<!DOCTYPE html>
<html class="writer-html5" lang="en" data-content_root="../../">
<head>
  <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Algorithms &mdash; SystemDS 3.2.0-dev documentation</title>
      <link rel="stylesheet" type="text/css" href="../../static/pygments.css?v=80d5e7a1" />
      <link rel="stylesheet" type="text/css" href="../../static/css/theme.css?v=19f00094" />

  
  <!--[if lt IE 9]>
    <script src="../../static/js/html5shiv.min.js"></script>
  <![endif]-->
  
        <script src="../../static/jquery.js?v=5d32c60e"></script>
        <script src="../../static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
        <script src="../../static/documentation_options.js?v=596fb6d1"></script>
        <script src="../../static/doctools.js?v=888ff710"></script>
        <script src="../../static/sphinx_highlight.js?v=dc90522c"></script>
    <script src="../../static/js/theme.js"></script>
    <link rel="index" title="Index" href="../../genindex.html" />
    <link rel="search" title="Search" href="../../search.html" />
    <link rel="next" title="Matrix" href="node/matrix.html" />
    <link rel="prev" title="SystemDSContext" href="../context/systemds_context.html" /> 
</head>

<body class="wy-body-for-nav"> 
  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >

          
          
          <a href="../../index.html" class="icon icon-home">
            SystemDS
          </a>
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>
        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <p class="caption" role="heading"><span class="caption-text">Getting Started:</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../getting_started/install.html">Install SystemDS</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../getting_started/simple_examples.html">QuickStart</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Guides</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../guide/federated.html">Federated Environment</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../guide/algorithms_basics.html">Built-in Algorithms</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../guide/python_end_to_end_tut.html">Python end-to-end tutorial</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">API</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../context/systemds_context.html">SystemDSContext</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Algorithms</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.WoE"><code class="docutils literal notranslate"><span class="pre">WoE()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.WoEApply"><code class="docutils literal notranslate"><span class="pre">WoEApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.abstain"><code class="docutils literal notranslate"><span class="pre">abstain()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.als"><code class="docutils literal notranslate"><span class="pre">als()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.alsCG"><code class="docutils literal notranslate"><span class="pre">alsCG()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.alsDS"><code class="docutils literal notranslate"><span class="pre">alsDS()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.alsPredict"><code class="docutils literal notranslate"><span class="pre">alsPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.alsTopkPredict"><code class="docutils literal notranslate"><span class="pre">alsTopkPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.apply_pipeline"><code class="docutils literal notranslate"><span class="pre">apply_pipeline()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.arima"><code class="docutils literal notranslate"><span class="pre">arima()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.auc"><code class="docutils literal notranslate"><span class="pre">auc()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.autoencoder_2layer"><code class="docutils literal notranslate"><span class="pre">autoencoder_2layer()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.bandit"><code class="docutils literal notranslate"><span class="pre">bandit()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.bivar"><code class="docutils literal notranslate"><span class="pre">bivar()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.components"><code class="docutils literal notranslate"><span class="pre">components()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.confusionMatrix"><code class="docutils literal notranslate"><span class="pre">confusionMatrix()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.cor"><code class="docutils literal notranslate"><span class="pre">cor()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.correctTypos"><code class="docutils literal notranslate"><span class="pre">correctTypos()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.correctTyposApply"><code class="docutils literal notranslate"><span class="pre">correctTyposApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.cox"><code class="docutils literal notranslate"><span class="pre">cox()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.cspline"><code class="docutils literal notranslate"><span class="pre">cspline()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.csplineCG"><code class="docutils literal notranslate"><span class="pre">csplineCG()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.csplineDS"><code class="docutils literal notranslate"><span class="pre">csplineDS()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.cvlm"><code class="docutils literal notranslate"><span class="pre">cvlm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.dbscan"><code class="docutils literal notranslate"><span class="pre">dbscan()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.dbscanApply"><code class="docutils literal notranslate"><span class="pre">dbscanApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.decisionTree"><code class="docutils literal notranslate"><span class="pre">decisionTree()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.decisionTreePredict"><code class="docutils literal notranslate"><span class="pre">decisionTreePredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.deepWalk"><code class="docutils literal notranslate"><span class="pre">deepWalk()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.denialConstraints"><code class="docutils literal notranslate"><span class="pre">denialConstraints()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.differenceStatistics"><code class="docutils literal notranslate"><span class="pre">differenceStatistics()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.discoverFD"><code class="docutils literal notranslate"><span class="pre">discoverFD()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.dist"><code class="docutils literal notranslate"><span class="pre">dist()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.dmv"><code class="docutils literal notranslate"><span class="pre">dmv()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.ema"><code class="docutils literal notranslate"><span class="pre">ema()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.executePipeline"><code class="docutils literal notranslate"><span class="pre">executePipeline()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.ffPredict"><code class="docutils literal notranslate"><span class="pre">ffPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.ffTrain"><code class="docutils literal notranslate"><span class="pre">ffTrain()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.fit_pipeline"><code class="docutils literal notranslate"><span class="pre">fit_pipeline()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.fixInvalidLengths"><code class="docutils literal notranslate"><span class="pre">fixInvalidLengths()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.fixInvalidLengthsApply"><code class="docutils literal notranslate"><span class="pre">fixInvalidLengthsApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.flattenQuantile"><code class="docutils literal notranslate"><span class="pre">flattenQuantile()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.frameSort"><code class="docutils literal notranslate"><span class="pre">frameSort()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.frequencyEncode"><code class="docutils literal notranslate"><span class="pre">frequencyEncode()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.frequencyEncodeApply"><code class="docutils literal notranslate"><span class="pre">frequencyEncodeApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.garch"><code class="docutils literal notranslate"><span class="pre">garch()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.gaussianClassifier"><code class="docutils literal notranslate"><span class="pre">gaussianClassifier()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.getAccuracy"><code class="docutils literal notranslate"><span class="pre">getAccuracy()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.glm"><code class="docutils literal notranslate"><span class="pre">glm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.glmPredict"><code class="docutils literal notranslate"><span class="pre">glmPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.gmm"><code class="docutils literal notranslate"><span class="pre">gmm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.gmmPredict"><code class="docutils literal notranslate"><span class="pre">gmmPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.gnmf"><code class="docutils literal notranslate"><span class="pre">gnmf()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.gridSearch"><code class="docutils literal notranslate"><span class="pre">gridSearch()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.hospitalResidencyMatch"><code class="docutils literal notranslate"><span class="pre">hospitalResidencyMatch()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.hyperband"><code class="docutils literal notranslate"><span class="pre">hyperband()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_brightness"><code class="docutils literal notranslate"><span class="pre">img_brightness()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_brightness_linearized"><code class="docutils literal notranslate"><span class="pre">img_brightness_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_crop"><code class="docutils literal notranslate"><span class="pre">img_crop()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_crop_linearized"><code class="docutils literal notranslate"><span class="pre">img_crop_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_cutout"><code class="docutils literal notranslate"><span class="pre">img_cutout()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_cutout_linearized"><code class="docutils literal notranslate"><span class="pre">img_cutout_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_invert"><code class="docutils literal notranslate"><span class="pre">img_invert()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_invert_linearized"><code class="docutils literal notranslate"><span class="pre">img_invert_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_mirror"><code class="docutils literal notranslate"><span class="pre">img_mirror()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_mirror_linearized"><code class="docutils literal notranslate"><span class="pre">img_mirror_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_posterize"><code class="docutils literal notranslate"><span class="pre">img_posterize()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_posterize_linearized"><code class="docutils literal notranslate"><span class="pre">img_posterize_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_rotate"><code class="docutils literal notranslate"><span class="pre">img_rotate()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_rotate_linearized"><code class="docutils literal notranslate"><span class="pre">img_rotate_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_sample_pairing"><code class="docutils literal notranslate"><span class="pre">img_sample_pairing()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_sample_pairing_linearized"><code class="docutils literal notranslate"><span class="pre">img_sample_pairing_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_shear"><code class="docutils literal notranslate"><span class="pre">img_shear()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_shear_linearized"><code class="docutils literal notranslate"><span class="pre">img_shear_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_transform"><code class="docutils literal notranslate"><span class="pre">img_transform()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_transform_linearized"><code class="docutils literal notranslate"><span class="pre">img_transform_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_translate"><code class="docutils literal notranslate"><span class="pre">img_translate()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.img_translate_linearized"><code class="docutils literal notranslate"><span class="pre">img_translate_linearized()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.impurityMeasures"><code class="docutils literal notranslate"><span class="pre">impurityMeasures()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByFD"><code class="docutils literal notranslate"><span class="pre">imputeByFD()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByFDApply"><code class="docutils literal notranslate"><span class="pre">imputeByFDApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByMean"><code class="docutils literal notranslate"><span class="pre">imputeByMean()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByMeanApply"><code class="docutils literal notranslate"><span class="pre">imputeByMeanApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByMedian"><code class="docutils literal notranslate"><span class="pre">imputeByMedian()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByMedianApply"><code class="docutils literal notranslate"><span class="pre">imputeByMedianApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByMode"><code class="docutils literal notranslate"><span class="pre">imputeByMode()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.imputeByModeApply"><code class="docutils literal notranslate"><span class="pre">imputeByModeApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.intersect"><code class="docutils literal notranslate"><span class="pre">intersect()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.km"><code class="docutils literal notranslate"><span class="pre">km()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.kmeans"><code class="docutils literal notranslate"><span class="pre">kmeans()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.kmeansPredict"><code class="docutils literal notranslate"><span class="pre">kmeansPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.knn"><code class="docutils literal notranslate"><span class="pre">knn()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.knnGraph"><code class="docutils literal notranslate"><span class="pre">knnGraph()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.knnbf"><code class="docutils literal notranslate"><span class="pre">knnbf()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.l2svm"><code class="docutils literal notranslate"><span class="pre">l2svm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.l2svmPredict"><code class="docutils literal notranslate"><span class="pre">l2svmPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lasso"><code class="docutils literal notranslate"><span class="pre">lasso()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lenetPredict"><code class="docutils literal notranslate"><span class="pre">lenetPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lenetTrain"><code class="docutils literal notranslate"><span class="pre">lenetTrain()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lm"><code class="docutils literal notranslate"><span class="pre">lm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lmCG"><code class="docutils literal notranslate"><span class="pre">lmCG()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lmDS"><code class="docutils literal notranslate"><span class="pre">lmDS()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lmPredict"><code class="docutils literal notranslate"><span class="pre">lmPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.lmPredictStats"><code class="docutils literal notranslate"><span class="pre">lmPredictStats()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.logSumExp"><code class="docutils literal notranslate"><span class="pre">logSumExp()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.mae"><code class="docutils literal notranslate"><span class="pre">mae()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.mape"><code class="docutils literal notranslate"><span class="pre">mape()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.matrixProfile"><code class="docutils literal notranslate"><span class="pre">matrixProfile()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.mcc"><code class="docutils literal notranslate"><span class="pre">mcc()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.mdedup"><code class="docutils literal notranslate"><span class="pre">mdedup()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.mice"><code class="docutils literal notranslate"><span class="pre">mice()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.miceApply"><code class="docutils literal notranslate"><span class="pre">miceApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.mse"><code class="docutils literal notranslate"><span class="pre">mse()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.msmape"><code class="docutils literal notranslate"><span class="pre">msmape()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.msvm"><code class="docutils literal notranslate"><span class="pre">msvm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.msvmPredict"><code class="docutils literal notranslate"><span class="pre">msvmPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.multiLogReg"><code class="docutils literal notranslate"><span class="pre">multiLogReg()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.multiLogRegPredict"><code class="docutils literal notranslate"><span class="pre">multiLogRegPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.na_locf"><code class="docutils literal notranslate"><span class="pre">na_locf()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.naiveBayes"><code class="docutils literal notranslate"><span class="pre">naiveBayes()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.naiveBayesPredict"><code class="docutils literal notranslate"><span class="pre">naiveBayesPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.normalize"><code class="docutils literal notranslate"><span class="pre">normalize()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.normalizeApply"><code class="docutils literal notranslate"><span class="pre">normalizeApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.nrmse"><code class="docutils literal notranslate"><span class="pre">nrmse()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.outlier"><code class="docutils literal notranslate"><span class="pre">outlier()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.outlierByArima"><code class="docutils literal notranslate"><span class="pre">outlierByArima()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.outlierByIQR"><code class="docutils literal notranslate"><span class="pre">outlierByIQR()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.outlierByIQRApply"><code class="docutils literal notranslate"><span class="pre">outlierByIQRApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.outlierBySd"><code class="docutils literal notranslate"><span class="pre">outlierBySd()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.outlierBySdApply"><code class="docutils literal notranslate"><span class="pre">outlierBySdApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.pca"><code class="docutils literal notranslate"><span class="pre">pca()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.pcaInverse"><code class="docutils literal notranslate"><span class="pre">pcaInverse()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.pcaTransform"><code class="docutils literal notranslate"><span class="pre">pcaTransform()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.pnmf"><code class="docutils literal notranslate"><span class="pre">pnmf()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.ppca"><code class="docutils literal notranslate"><span class="pre">ppca()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.psnr"><code class="docutils literal notranslate"><span class="pre">psnr()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.randomForest"><code class="docutils literal notranslate"><span class="pre">randomForest()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.randomForestPredict"><code class="docutils literal notranslate"><span class="pre">randomForestPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.rmse"><code class="docutils literal notranslate"><span class="pre">rmse()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.scale"><code class="docutils literal notranslate"><span class="pre">scale()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.scaleApply"><code class="docutils literal notranslate"><span class="pre">scaleApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.scaleMinMax"><code class="docutils literal notranslate"><span class="pre">scaleMinMax()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.selectByVarThresh"><code class="docutils literal notranslate"><span class="pre">selectByVarThresh()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.setdiff"><code class="docutils literal notranslate"><span class="pre">setdiff()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.sherlock"><code class="docutils literal notranslate"><span class="pre">sherlock()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.sherlockPredict"><code class="docutils literal notranslate"><span class="pre">sherlockPredict()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.shortestPath"><code class="docutils literal notranslate"><span class="pre">shortestPath()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.sigmoid"><code class="docutils literal notranslate"><span class="pre">sigmoid()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.skewness"><code class="docutils literal notranslate"><span class="pre">skewness()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.slicefinder"><code class="docutils literal notranslate"><span class="pre">slicefinder()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.smape"><code class="docutils literal notranslate"><span class="pre">smape()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.smote"><code class="docutils literal notranslate"><span class="pre">smote()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.softmax"><code class="docutils literal notranslate"><span class="pre">softmax()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.split"><code class="docutils literal notranslate"><span class="pre">split()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.splitBalanced"><code class="docutils literal notranslate"><span class="pre">splitBalanced()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.stableMarriage"><code class="docutils literal notranslate"><span class="pre">stableMarriage()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.statsNA"><code class="docutils literal notranslate"><span class="pre">statsNA()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.steplm"><code class="docutils literal notranslate"><span class="pre">steplm()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.stratstats"><code class="docutils literal notranslate"><span class="pre">stratstats()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.symmetricDifference"><code class="docutils literal notranslate"><span class="pre">symmetricDifference()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.tSNE"><code class="docutils literal notranslate"><span class="pre">tSNE()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.toOneHot"><code class="docutils literal notranslate"><span class="pre">toOneHot()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.tomeklink"><code class="docutils literal notranslate"><span class="pre">tomeklink()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.topk_cleaning"><code class="docutils literal notranslate"><span class="pre">topk_cleaning()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.underSampling"><code class="docutils literal notranslate"><span class="pre">underSampling()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.union"><code class="docutils literal notranslate"><span class="pre">union()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.univar"><code class="docutils literal notranslate"><span class="pre">univar()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.vectorToCsv"><code class="docutils literal notranslate"><span class="pre">vectorToCsv()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.winsorize"><code class="docutils literal notranslate"><span class="pre">winsorize()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.winsorizeApply"><code class="docutils literal notranslate"><span class="pre">winsorizeApply()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.xdummy1"><code class="docutils literal notranslate"><span class="pre">xdummy1()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.xdummy2"><code class="docutils literal notranslate"><span class="pre">xdummy2()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.xgboost"><code class="docutils literal notranslate"><span class="pre">xgboost()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.xgboostPredictClassification"><code class="docutils literal notranslate"><span class="pre">xgboostPredictClassification()</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#systemds.operator.algorithm.xgboostPredictRegression"><code class="docutils literal notranslate"><span class="pre">xgboostPredictRegression()</span></code></a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="node/matrix.html">Matrix</a></li>
<li class="toctree-l1"><a class="reference internal" href="node/frame.html">Frame</a></li>
<li class="toctree-l1"><a class="reference internal" href="node/list.html">List</a></li>
<li class="toctree-l1"><a class="reference internal" href="node/scalar.html">Scalar</a></li>
<li class="toctree-l1"><a class="reference internal" href="node/source.html">Source</a></li>
</ul>
<p class="caption" role="heading"><span class="caption-text">Internals API</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="operation_node.html">Operation Node</a></li>
<li class="toctree-l1"><a class="reference internal" href="../script_building/dag.html">Dag</a></li>
<li class="toctree-l1"><a class="reference internal" href="../script_building/script.html">Script</a></li>
<li class="toctree-l1"><a class="reference internal" href="../utils/converters.html">Converters</a></li>
<li class="toctree-l1"><a class="reference internal" href="../utils/helpers.html">Helpers</a></li>
</ul>

        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../index.html">SystemDS</a>
      </nav>

      <div class="wy-nav-content">
        <div class="rst-content">
          <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../index.html" class="icon icon-home" aria-label="Home"></a></li>
      <li class="breadcrumb-item active">Algorithms</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../sources/api/operator/algorithms.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
             
  <section id="algorithms">
<h1>Algorithms<a class="headerlink" href="#algorithms" title="Link to this heading"></a></h1>
<p>SystemDS support different Machine learning algorithms out of the box.</p>
<p>As an example the lm algorithm can be used as follows:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Import numpy and SystemDS</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">systemds.context</span> <span class="kn">import</span> <span class="n">SystemDSContext</span>
<span class="kn">from</span> <span class="nn">systemds.operator.algorithm</span> <span class="kn">import</span> <span class="n">lm</span>

<span class="c1"># Set a seed</span>
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="c1"># Generate matrix of feature vectors</span>
<span class="n">features</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">15</span><span class="p">)</span>
<span class="c1"># Generate a 1-column matrix of response values</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>

<span class="c1"># compute the weights</span>
<span class="k">with</span> <span class="n">SystemDSContext</span><span class="p">()</span> <span class="k">as</span> <span class="n">sds</span><span class="p">:</span>
  <span class="n">weights</span> <span class="o">=</span> <span class="n">lm</span><span class="p">(</span><span class="n">sds</span><span class="o">.</span><span class="n">from_numpy</span><span class="p">(</span><span class="n">features</span><span class="p">),</span> <span class="n">sds</span><span class="o">.</span><span class="n">from_numpy</span><span class="p">(</span><span class="n">y</span><span class="p">))</span><span class="o">.</span><span class="n">compute</span><span class="p">()</span>
  <span class="nb">print</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span>
</pre></div>
</div>
<p>The output should be similar to</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">[[</span><span class="o">-</span><span class="mf">0.11538199</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="mf">0.20386541</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="mf">0.39956035</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">1.04078623</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">0.4327084</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">0.18954599</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">0.49858968</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="mf">0.26812763</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">0.09961844</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="mf">0.57000751</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="mf">0.43386048</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">0.55358873</span><span class="p">]</span>
<span class="p">[</span><span class="o">-</span><span class="mf">0.54638565</span><span class="p">]</span>
<span class="p">[</span> <span class="mf">0.2205885</span> <span class="p">]</span>
<span class="p">[</span> <span class="mf">0.37957689</span><span class="p">]]</span>
</pre></div>
</div>
<dl class="py function" id="module-systemds.operator.algorithm">
<dt class="sig sig-object py" id="systemds.operator.algorithm.WoE">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">WoE</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.WoE" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>function Weight of evidence / information gain</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – <p>—</p>
</p></li>
<li><p><strong>Y</strong> – <p>—</p>
</p></li>
<li><p><strong>mask</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Weighted X matrix where the entropy mask is applied</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>A entropy matrix to apply to data</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.WoEApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">WoEApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">entropyMatrix</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.WoEApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>function Weight of evidence / information gain apply on new data</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – <p>—</p>
</p></li>
<li><p><strong>Y</strong> – <p>—</p>
</p></li>
<li><p><strong>entropyMatrix</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Weighted X matrix where the entropy mask is applied</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.abstain">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">abstain</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">threshold</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.abstain" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function calls the multiLogReg-function in which solves Multinomial
Logistic Regression using Trust Region method</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – matrix of feature vectors</p></li>
<li><p><strong>Y</strong> – matrix with category labels</p></li>
<li><p><strong>threshold</strong> – threshold to clear otherwise return X and Y unmodified</p></li>
<li><p><strong>verbose</strong> – flag specifying if logging information should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>abstained output X</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>abstained output Y</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.als">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">als</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.als" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script computes an approximate factorization of a low-rank matrix X into two matrices U and V
using different implementations of the Alternating-Least-Squares (ALS) algorithm.
Matrices U and V are computed by minimizing a loss function (with regularization).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location to read the input matrix X to be factorized</p></li>
<li><p><strong>rank</strong> – Rank of the factorization</p></li>
<li><p><strong>regType</strong> – Regularization:
“L2” = L2 regularization;
f (U, V) = 0.5 * sum (W * (U %*% V - X) ^ 2)
+ 0.5 * reg * (sum (U ^ 2) + sum (V ^ 2))
“wL2” = weighted L2 regularization
f (U, V) = 0.5 * sum (W * (U %*% V - X) ^ 2)
+ 0.5 * reg * (sum (U ^ 2 * row_nonzeros)
+ sum (V ^ 2 * col_nonzeros))</p></li>
<li><p><strong>reg</strong> – Regularization parameter, no regularization if 0.0</p></li>
<li><p><strong>maxi</strong> – Maximum number of iterations</p></li>
<li><p><strong>check</strong> – Check for convergence after every iteration, i.e., updating U and V once</p></li>
<li><p><strong>thr</strong> – Assuming check is set to TRUE, the algorithm stops and convergence is declared
if the decrease in loss in any two consecutive iterations falls below this threshold;
if check is FALSE thr is ignored</p></li>
<li><p><strong>seed</strong> – The seed to random parts of the algorithm</p></li>
<li><p><strong>verbose</strong> – If the algorithm should run verbosely</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>An m x r matrix where r is the factorization rank</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>An m x r matrix where r is the factorization rank</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.alsCG">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">alsCG</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.alsCG" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script computes an approximate factorization of a low-rank matrix X into two matrices U and V
using the Alternating-Least-Squares (ALS) algorithm with conjugate gradient.
Matrices U and V are computed by minimizing a loss function (with regularization).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location to read the input matrix X to be factorized</p></li>
<li><p><strong>rank</strong> – Rank of the factorization</p></li>
<li><p><strong>regType</strong> – Regularization:
“L2” = L2 regularization;
f (U, V) = 0.5 * sum (W * (U %*% V - X) ^ 2)
+ 0.5 * reg * (sum (U ^ 2) + sum (V ^ 2))
“wL2” = weighted L2 regularization
f (U, V) = 0.5 * sum (W * (U %*% V - X) ^ 2)
+ 0.5 * reg * (sum (U ^ 2 * row_nonzeros)
+ sum (V ^ 2 * col_nonzeros))</p></li>
<li><p><strong>reg</strong> – Regularization parameter, no regularization if 0.0</p></li>
<li><p><strong>maxi</strong> – Maximum number of iterations</p></li>
<li><p><strong>check</strong> – Check for convergence after every iteration, i.e., updating U and V once</p></li>
<li><p><strong>thr</strong> – Assuming check is set to TRUE, the algorithm stops and convergence is declared
if the decrease in loss in any two consecutive iterations falls below this threshold;
if check is FALSE thr is ignored</p></li>
<li><p><strong>seed</strong> – The seed to random parts of the algorithm</p></li>
<li><p><strong>verbose</strong> – If the algorithm should run verbosely</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>An m x r matrix where r is the factorization rank</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>An m x r matrix where r is the factorization rank</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.alsDS">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">alsDS</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.alsDS" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Alternating-Least-Squares (ALS) algorithm using a direct solve method for
individual least squares problems (reg=”L2”). This script computes an 
approximate factorization of a low-rank matrix V into two matrices L and R.
Matrices L and R are computed by minimizing a loss function (with regularization).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location to read the input matrix V to be factorized</p></li>
<li><p><strong>rank</strong> – Rank of the factorization</p></li>
<li><p><strong>reg</strong> – Regularization parameter, no regularization if 0.0</p></li>
<li><p><strong>maxi</strong> – Maximum number of iterations</p></li>
<li><p><strong>check</strong> – Check for convergence after every iteration, i.e., updating L and R once</p></li>
<li><p><strong>thr</strong> – Assuming check is set to TRUE, the algorithm stops and convergence is declared
if the decrease in loss in any two consecutive iterations falls below this threshold;
if check is FALSE thr is ignored</p></li>
<li><p><strong>seed</strong> – The seed to random parts of the algorithm</p></li>
<li><p><strong>verbose</strong> – If the algorithm should run verbosely</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>An m x r matrix where r is the factorization rank</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>An m x r matrix where r is the factorization rank</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.alsPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">alsPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">userIDs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">I</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">L</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">R</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.alsPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script computes the rating/scores for a given list of userIDs 
using 2 factor matrices L and R. We assume that all users have rates 
at least once and all items have been rates at least once.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>userIDs</strong> – Column vector of user-ids (n x 1)</p></li>
<li><p><strong>I</strong> – Indicator matrix user-id x user-id to exclude from scoring</p></li>
<li><p><strong>L</strong> – The factor matrix L: user-id x feature-id</p></li>
<li><p><strong>R</strong> – The factor matrix R: feature-id x item-id</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The output user-id/item-id/score#</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.alsTopkPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">alsTopkPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">userIDs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">I</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">L</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">R</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.alsTopkPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script computes the top-K rating/scores for a given list of userIDs 
using 2 factor matrices L and R. We assume that all users have rates 
at least once and all items have been rates at least once.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>userIDs</strong> – Column vector of user-ids (n x 1)</p></li>
<li><p><strong>I</strong> – Indicator matrix user-id x user-id to exclude from scoring</p></li>
<li><p><strong>L</strong> – The factor matrix L: user-id x feature-id</p></li>
<li><p><strong>R</strong> – The factor matrix R: feature-id x item-id</p></li>
<li><p><strong>K</strong> – The number of top-K items</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>A matrix containing the top-K item-ids with highest predicted ratings for the specified users (rows)</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>A matrix containing the top-K predicted ratings for the specified users (rows)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.apply_pipeline">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">apply_pipeline</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">testData</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pip</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">applyFunc</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">hp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">exState</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">iState</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.apply_pipeline" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script will read the dirty and clean data, then it will apply the best pipeline on dirty data
and then will classify both cleaned dataset and check if the cleaned dataset is performing same as original dataset
in terms of classification accuracy</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>trainData</strong> – <p>—</p>
</p></li>
<li><p><strong>testData</strong> – <p>—</p>
</p></li>
<li><p><strong>metaData</strong> – <p>—</p>
</p></li>
<li><p><strong>lp</strong> – <p>—</p>
</p></li>
<li><p><strong>pip</strong> – <p>—</p>
</p></li>
<li><p><strong>hp</strong> – <p>—</p>
</p></li>
<li><p><strong>evaluationFunc</strong> – <p>—</p>
</p></li>
<li><p><strong>evalFunHp</strong> – <p>—</p>
</p></li>
<li><p><strong>isLastLabel</strong> – <p>—</p>
</p></li>
<li><p><strong>correctTypos</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.arima">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">arima</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.arima" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that implements ARIMA</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – The input Matrix to apply Arima on.</p></li>
<li><p><strong>max_func_invoc</strong> – <p>—</p>
</p></li>
<li><p><strong>p</strong> – non-seasonal AR order</p></li>
<li><p><strong>d</strong> – non-seasonal differencing order</p></li>
<li><p><strong>q</strong> – non-seasonal MA order</p></li>
<li><p><strong>P</strong> – seasonal AR order</p></li>
<li><p><strong>D</strong> – seasonal differencing order</p></li>
<li><p><strong>Q</strong> – seasonal MA order</p></li>
<li><p><strong>s</strong> – period in terms of number of time-steps</p></li>
<li><p><strong>include_mean</strong> – center to mean 0, and include in result</p></li>
<li><p><strong>solver</strong> – solver, is either “cg” or “jacobi”</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The calculated coefficients</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.auc">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">auc</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">P</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.auc" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function computes the area under the ROC curve (AUC)
for binary classifiers.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>Y</strong> – Binary response vector (shape: n x 1), in -1/+1 or 0/1 encoding</p></li>
<li><p><strong>P</strong> – Prediction scores (predictor such as estimated probabilities)
for true class (shape: n x 1), assumed in [0,1]</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Area under the ROC curve (AUC)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.autoencoder_2layer">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">autoencoder_2layer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">num_hidden1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">num_hidden2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_epochs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.autoencoder_2layer" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Trains a 2-layer autoencoder with minibatch SGD and step-size decay.
If invoked with H1 &gt; H2 then it becomes a ‘bowtie’ structured autoencoder
Weights are initialized using Glorot &amp; Bengio (2010) AISTATS initialization.
The script standardizes the input before training (can be turned off).
Also, it randomly reshuffles rows before training.
Currently, tanh is set to be the activation function. 
By re-implementing ‘func’ DML-bodied function, one can change the activation.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Filename where the input is stored</p></li>
<li><p><strong>num_hidden1</strong> – Number of neurons in the 1st hidden layer</p></li>
<li><p><strong>num_hidden2</strong> – Number of neurons in the 2nd hidden layer</p></li>
<li><p><strong>max_epochs</strong> – Number of epochs to train for</p></li>
<li><p><strong>full_obj</strong> – If TRUE, Computes objective function value (squared-loss)
at the end of each epoch. Note that, computing the full
objective can take a lot of time.</p></li>
<li><p><strong>batch_size</strong> – Mini-batch size (training parameter)</p></li>
<li><p><strong>step</strong> – Initial step size (training parameter)</p></li>
<li><p><strong>decay</strong> – Decays step size after each epoch (training parameter)</p></li>
<li><p><strong>mu</strong> – Momentum parameter (training parameter)</p></li>
<li><p><strong>W1_rand</strong> – Weights might be initialized via input matrices</p></li>
<li><p><strong>W2_rand</strong> – <p>—</p>
</p></li>
<li><p><strong>W3_rand</strong> – <p>—</p>
</p></li>
<li><p><strong>W4_rand</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix storing weights between input layer and 1st hidden layer</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix storing bias between input layer and 1st hidden layer</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix storing weights between 1st hidden layer and 2nd hidden layer</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix storing bias between 1st hidden layer and 2nd hidden layer</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix storing weights between 2nd hidden layer and 3rd hidden layer</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix storing bias between 2nd hidden layer and 3rd hidden layer</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix storing weights between 3rd hidden layer and output layer</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix storing bias between 3rd hidden layer and output layer</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix storing the hidden (2nd) layer representation if needed</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.bandit">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">bandit</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X_train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y_train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">X_test</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y_test</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">metaList</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">evaluationFunc</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">evalFunHp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">lp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">lpHp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">primitives</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">param</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">baseLineScore</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">cv</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.bandit" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>In The bandit function the objective is to find an arm that optimizes
a known functional of the unknown arm-reward distributions.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X_train</strong> – <p>—</p>
</p></li>
<li><p><strong>Y_train</strong> – <p>—</p>
</p></li>
<li><p><strong>X_test</strong> – <p>—</p>
</p></li>
<li><p><strong>Y_test</strong> – <p>—</p>
</p></li>
<li><p><strong>metaList</strong> – <p>—</p>
</p></li>
<li><p><strong>evaluationFunc</strong> – <p>—</p>
</p></li>
<li><p><strong>evalFunHp</strong> – <p>—</p>
</p></li>
<li><p><strong>lp</strong> – <p>—</p>
</p></li>
<li><p><strong>primitives</strong> – <p>—</p>
</p></li>
<li><p><strong>params</strong> – <p>—</p>
</p></li>
<li><p><strong>K</strong> – <p>—</p>
</p></li>
<li><p><strong>R</strong> – <p>—</p>
</p></li>
<li><p><strong>baseLineScore</strong> – <p>—</p>
</p></li>
<li><p><strong>cv</strong> – <p>—</p>
</p></li>
<li><p><strong>cvk</strong> – <p>—</p>
</p></li>
<li><p><strong>verbose</strong> – <p>—</p>
</p></li>
<li><p><strong>output</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.bivar">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">bivar</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">S1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">S2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">T1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">T2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.bivar" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>For a given pair of attribute sets, compute bivariate statistics between all attribute pairs.
Given, index1 = {A_11, A_12, … A_1m} and index2 = {A_21, A_22, … A_2n}
compute bivariate stats for m*n pairs (A_1i, A_2j), (1&lt;= i &lt;=m) and (1&lt;= j &lt;=n).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input matrix</p></li>
<li><p><strong>S1</strong> – First attribute set {A_11, A_12, … A_1m}</p></li>
<li><p><strong>S2</strong> – Second attribute set {A_21, A_22, … A_2n}</p></li>
<li><p><strong>T1</strong> – Kind for attributes in S1
(kind=1 for scale, kind=2 for nominal, kind=3 for ordinal)</p></li>
<li><p><strong>verbose</strong> – Print bivar stats</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>basestats_scale_scale as output with bivar stats</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>basestats_nominal_scale as output with bivar stats</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>basestats_nominal_nominal as output with bivar stats</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>basestats_ordinal_ordinal as output with bivar stats</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.components">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">components</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">G</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.components" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Computes the connected components of a graph and returns a
vector indicating the assignment of vertices to components,
where each component is identified by the maximum vertex ID
(i.e., row/column position of the input graph)</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location to read the matrix of feature vectors</p></li>
<li><p><strong>Y</strong> – Location to read the matrix with category labels</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling X columns: 0 = no intercept,
no shifting, no rescaling; 1 = add intercept, but neither shift nor rescale X;
2 = add intercept, shift &amp; rescale X columns to mean = 0, variance = 1</p></li>
<li><p><strong>tol</strong> – tolerance (“epsilon”)</p></li>
<li><p><strong>reg</strong> – regularization parameter (lambda = 1/C); intercept is not regularized</p></li>
<li><p><strong>maxi</strong> – max. number of outer (Newton) iterations</p></li>
<li><p><strong>maxii</strong> – max. number of inner (conjugate gradient) iterations, 0 = no max</p></li>
<li><p><strong>verbose</strong> – flag specifying if logging information should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>regression betas as output for prediction</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.confusionMatrix">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">confusionMatrix</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">P</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.confusionMatrix" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Accepts a vector for prediction and a one-hot-encoded matrix
Then it computes the max value of each vector and compare them
After which, it calculates and returns the sum of classifications
and the average of each true class.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>                <span class="kc">True</span> <span class="n">Labels</span>
                  <span class="mi">1</span>    <span class="mi">2</span>
              <span class="mi">1</span>   <span class="n">TP</span> <span class="o">|</span> <span class="n">FP</span>
<span class="n">Predictions</span>      <span class="o">----+----</span>
              <span class="mi">2</span>   <span class="n">FN</span> <span class="o">|</span> <span class="n">TN</span>
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>P</strong> – vector of Predictions</p></li>
<li><p><strong>Y</strong> – vector of Golden standard One Hot Encoded; the one hot
encoded vector of actual labels</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The Confusion Matrix Sums of classifications</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>The Confusion Matrix averages of each true class</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.cor">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">cor</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.cor" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This Function compute correlation matrix</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – A Matrix Input to compute the correlation on</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Correlation matrix of the input matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.correctTypos">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">correctTypos</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">strings</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.correctTypos" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Corrects corrupted frames of strings
This algorithm operates on the assumption that most strings are correct
and simply swaps strings that do not occur often with similar strings that 
occur more often</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>References:
Fred J. Damerau. 1964. 
  A technique for computer detection and correction of spelling errors. 
  Commun. ACM 7, 3 (March 1964), 171–176. 
  DOI:https://doi.org/10.1145/363958.363994
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>strings</strong> – The nx1 input frame of corrupted strings</p></li>
<li><p><strong>frequency_threshold</strong> – Strings that occur above this frequency level will not be corrected</p></li>
<li><p><strong>distance_threshold</strong> – Max distance at which strings are considered similar</p></li>
<li><p><strong>is_verbose</strong> – Print debug information</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Corrected nx1 output frame</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.correctTyposApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">correctTyposApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">strings</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">distance_matrix</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">dict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.correctTyposApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Corrects corrupted frames of strings
This algorithm operates on the assumption that most strings are correct
and simply swaps strings that do not occur often with similar strings that 
occur more often</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>References:
Fred J. Damerau. 1964. 
  A technique for computer detection and correction of spelling errors. 
  Commun. ACM 7, 3 (March 1964), 171–176. 
  DOI:https://doi.org/10.1145/363958.363994
</pre></div>
</div>
<p>TODO: future: add parameter for list of words that are sure to be correct</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>strings</strong> – The nx1 input frame of corrupted strings</p></li>
<li><p><strong>nullMask</strong> – <p>—</p>
</p></li>
<li><p><strong>frequency_threshold</strong> – Strings that occur above this frequency level will not be corrected</p></li>
<li><p><strong>distance_threshold</strong> – Max distance at which strings are considered similar</p></li>
<li><p><strong>matrix</strong> (<em>distance</em>) – <p>—</p>
</p></li>
<li><p><strong>dict</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Corrected nx1 output frame</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.cox">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">cox</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">TE</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">F</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">R</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.cox" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script fits a cox Proportional hazard regression model.
The Breslow method is used for handling ties and the regression parameters 
are computed using trust region newton method with conjugate gradient</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location to read the input matrix X containing the survival data
containing the following information
1: timestamps
2: whether an event occurred (1) or data is censored (0)
3: feature vectors</p></li>
<li><p><strong>TE</strong> – Column indices of X as a column vector which contain timestamp
(first row) and event information (second row)</p></li>
<li><p><strong>F</strong> – Column indices of X as a column vector which are to be used for
fitting the Cox model</p></li>
<li><p><strong>R</strong> – If factors (categorical variables) are available in the input matrix
X, location to read matrix R containing the start and end indices of
the factors in X
R[,1]: start indices
R[,2]: end indices
Alternatively, user can specify the indices of the baseline level of
each factor which needs to be removed from X; in this case the start
and end indices corresponding to the baseline level need to be the same;
if R is not provided by default all variables are considered to be continuous</p></li>
<li><p><strong>alpha</strong> – Parameter to compute a 100*(1-alpha)% confidence interval for the betas</p></li>
<li><p><strong>tol</strong> – Tolerance (“epsilon”)</p></li>
<li><p><strong>moi</strong> – Max. number of outer (Newton) iterations</p></li>
<li><p><strong>mii</strong> – Max. number of inner (conjugate gradient) iterations, 0 = no max</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>A D x 7 matrix M, where D denotes the number of covariates, with the following schema:
M[,1]: betas
M[,2]: exp(betas)
M[,3]: standard error of betas
M[,4]: Z
M[,5]: P-value
M[,6]: lower 100*(1-alpha)% confidence interval of betas
M[,7]: upper 100*(1-alpha)% confidence interval of betas</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Two matrices containing a summary of some statistics of the fitted model:
1 - File S with the following format
- row 1: no. of observations
- row 2: no. of events
- row 3: log-likelihood
- row 4: AIC
- row 5: Rsquare (Cox &amp; Snell)
- row 6: max possible Rsquare
2 - File T with the following format
- row 1: Likelihood ratio test statistic, degree of freedom, P-value
- row 2: Wald test statistic, degree of freedom, P-value
- row 3: Score (log-rank) test statistic, degree of freedom, P-value</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Additionally, the following matrices are stored (needed for prediction)
1- A column matrix RT that contains the order-preserving recoded timestamps from X
2- Matrix XO which is matrix X with sorted timestamps
3- Variance-covariance matrix of the betas COV
4- A column matrix MF that contains the column indices of X with the baseline factors removed (if available)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.cspline">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">cspline</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">inp_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.cspline" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Solves Cubic Spline Interpolation</p>
<p>Algorithms: implement <a class="reference external" href="https://en.wikipedia.org/wiki/Spline_interpolation#Algorithm_to_find_the_interpolating_cubic_spline">https://en.wikipedia.org/wiki/Spline_interpolation#Algorithm_to_find_the_interpolating_cubic_spline</a>
It use natural spline with q1’’(x0) == qn’’(xn) == 0.0</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – 1-column matrix of x values knots. It is assumed that x values are
monotonically increasing and there is no duplicates points in X</p></li>
<li><p><strong>Y</strong> – 1-column matrix of corresponding y values knots</p></li>
<li><p><strong>inp_x</strong> – the given input x, for which the cspline will find predicted y</p></li>
<li><p><strong>mode</strong> – Specifies the method for cspline (DS - Direct Solve, CG - Conjugate Gradient)</p></li>
<li><p><strong>tol</strong> – Tolerance (epsilon); conjugate graduent procedure terminates early if
L2 norm of the beta-residual is less than tolerance * its initial norm</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations, 0 = no maximum</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Predicted value</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix of k parameters</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.csplineCG">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">csplineCG</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">inp_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.csplineCG" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin that solves cubic spline interpolation using conjugate gradient algorithm</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – 1-column matrix of x values knots. It is assumed that x values are
monotonically increasing and there is no duplicates points in X</p></li>
<li><p><strong>Y</strong> – 1-column matrix of corresponding y values knots</p></li>
<li><p><strong>inp_x</strong> – the given input x, for which the cspline will find predicted y.</p></li>
<li><p><strong>tol</strong> – Tolerance (epsilon); conjugate gradient procedure terminates early if
L2 norm of the beta-residual is less than tolerance * its initial norm</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations, 0 = no maximum</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Predicted value</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix of k parameters</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.csplineDS">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">csplineDS</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">inp_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.csplineDS" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin that solves cubic spline interpolation using a direct solver.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – 1-column matrix of x values knots. It is assumed that x values are
monotonically increasing and there is no duplicates points in X</p></li>
<li><p><strong>Y</strong> – 1-column matrix of corresponding y values knots</p></li>
<li><p><strong>inp_x</strong> – the given input x, for which the cspline will find predicted y.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Predicted value</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix of k parameters</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.cvlm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">cvlm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.cvlm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The cvlm-function is used for cross-validation of the provided data model. This function follows a non-exhaustive cross
validation method. It uses lm and lmPredict functions to solve the linear regression and to predict the class of a
feature vector with no intercept, shifting, and rescaling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Recorded Data set into matrix</p></li>
<li><p><strong>y</strong> – 1-column matrix of response values.</p></li>
<li><p><strong>k</strong> – Number of subsets needed, It should always be more than 1 and less than nrow(X)</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling the columns of X</p></li>
<li><p><strong>reg</strong> – Regularization constant (lambda) for L2-regularization. set to nonzero for
highly dependant/sparse/numerous features</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Response values</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Validated data set</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.dbscan">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">dbscan</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.dbscan" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements the DBSCAN clustering algorithm using an Euclidean
distance matrix.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – The input Matrix to do DBSCAN on.</p></li>
<li><p><strong>eps</strong> – Maximum distance between two points for one to
be considered reachable for the other.</p></li>
<li><p><strong>minPts</strong> – Number of points in a neighborhood for a point to
be considered as a core point
(includes the point itself).</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The clustering matrix</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>The cluster model</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.dbscanApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">dbscanApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">clusterModel</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">eps</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.dbscanApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements the outlier detection/prediction algorithm using a DBScan model</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – The input Matrix to do outlier detection on.</p></li>
<li><p><strong>clusterModel</strong> – Model of clusters to predict outliers against.</p></li>
<li><p><strong>eps</strong> – Maximum distance between two points for one to be considered reachable for the other.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Predicted outliers</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.decisionTree">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">decisionTree</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ctypes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.decisionTree" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script implements decision trees for recoded and binned categorical and
numerical input features. We train a single CART (classification and
regression tree) decision trees depending on the provided labels y, either
classification (majority vote per leaf) or regression (average per leaf).</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">For</span> <span class="n">example</span><span class="p">,</span> <span class="n">give</span> <span class="n">a</span> <span class="n">feature</span> <span class="n">matrix</span> <span class="k">with</span> <span class="n">features</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">,</span><span class="n">c</span><span class="p">,</span><span class="n">d</span><span class="p">]</span>
<span class="ow">and</span> <span class="n">the</span> <span class="n">following</span> <span class="n">trees</span><span class="p">,</span> <span class="n">M</span> <span class="n">would</span> <span class="n">look</span> <span class="k">as</span> <span class="n">follows</span><span class="p">:</span>

<span class="p">(</span><span class="n">L1</span><span class="p">)</span>               <span class="o">|</span><span class="n">d</span><span class="o">&lt;</span><span class="mi">5</span><span class="o">|</span>
                  <span class="o">/</span>            <span class="p">(</span><span class="n">L2</span><span class="p">)</span>           <span class="n">P1</span><span class="p">:</span><span class="mi">2</span>    <span class="o">|</span><span class="n">a</span><span class="o">&lt;</span><span class="mi">7</span><span class="o">|</span>
                       <span class="o">/</span>          <span class="p">(</span><span class="n">L3</span><span class="p">)</span>                 <span class="n">P2</span><span class="p">:</span><span class="mi">2</span> <span class="n">P3</span><span class="p">:</span><span class="mi">1</span>

<span class="o">--&gt;</span> <span class="n">M</span> <span class="o">:=</span>
<span class="p">[[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">]]</span>
 <span class="o">|</span><span class="p">(</span><span class="n">L1</span><span class="p">)</span><span class="o">|</span> <span class="o">|</span>  <span class="p">(</span><span class="n">L2</span><span class="p">)</span>   <span class="o">|</span> <span class="o">|</span>        <span class="p">(</span><span class="n">L3</span><span class="p">)</span>         <span class="o">|</span>
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix in recoded/binned representation</p></li>
<li><p><strong>y</strong> – Label matrix in recoded/binned representation</p></li>
<li><p><strong>ctypes</strong> – Row-Vector of column types [1 scale/ordinal, 2 categorical]
of shape 1-by-(ncol(X)+1), where the last entry is the y type</p></li>
<li><p><strong>max_depth</strong> – Maximum depth of the learned tree (stopping criterion)</p></li>
<li><p><strong>min_leaf</strong> – Minimum number of samples in leaf nodes (stopping criterion),
odd number recommended to avoid 50/50 leaf label decisions</p></li>
<li><p><strong>min_split</strong> – Minimum number of samples in leaf for attempting a split</p></li>
<li><p><strong>max_features</strong> – Parameter controlling the number of features used as split
candidates at tree nodes: m = ceil(num_features^max_features)</p></li>
<li><p><strong>max_values</strong> – Parameter controlling the number of values per feature used
as split candidates: nb = ceil(num_values^max_values)</p></li>
<li><p><strong>max_dataratio</strong> – Parameter in [0,1] controlling when to materialize data
subsets of X and y on node splits. When set to 0, we always
scan the original X and y, which has the benefit of avoiding
the allocation and maintenance of data for all active nodes.
When set to 0.01 we rematerialize whenever the sub-tree data
would be less than 1% of last the parent materialize data size.</p></li>
<li><p><strong>impurity</strong> – Impurity measure: entropy, gini (default), rss (regression)</p></li>
<li><p><strong>seed</strong> – Fixed seed for randomization of samples and split candidates</p></li>
<li><p><strong>verbose</strong> – Flag indicating verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix M containing the learned trees, in linearized form</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.decisionTreePredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">decisionTreePredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ctypes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">M</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.decisionTreePredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script implements random forest prediction for recoded and binned
categorical and numerical input features.
Hummingbird paper (<a class="reference external" href="https://www.usenix.org/system/files/osdi20-nakandala.pdf">https://www.usenix.org/system/files/osdi20-nakandala.pdf</a>).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix in recoded/binned representation</p></li>
<li><p><strong>y</strong> – Label matrix in recoded/binned representation,
optional for accuracy evaluation</p></li>
<li><p><strong>ctypes</strong> – Row-Vector of column types [1 scale/ordinal, 2 categorical]</p></li>
<li><p><strong>M</strong> – Matrix M holding the learned tree in linearized form
see decisionTree() for the detailed tree representation.</p></li>
<li><p><strong>strategy</strong> – Prediction strategy, can be one of [“GEMM”, “TT”, “PTT”],
referring to “Generic matrix multiplication”,
“Tree traversal”, and “Perfect tree traversal”, respectively</p></li>
<li><p><strong>verbose</strong> – Flag indicating verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Label vector of predictions</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.deepWalk">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">deepWalk</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">Graph</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">d</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gamma</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">t</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.deepWalk" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script performs DeepWalk on a given graph (<a class="reference external" href="https://arxiv.org/pdf/1403.6652.pdf">https://arxiv.org/pdf/1403.6652.pdf</a>)</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>Graph</strong> – adjacency matrix of a graph (n x n)</p></li>
<li><p><strong>w</strong> – window size</p></li>
<li><p><strong>d</strong> – embedding size</p></li>
<li><p><strong>gamma</strong> – walks per vertex</p></li>
<li><p><strong>t</strong> – walk length</p></li>
<li><p><strong>alpha</strong> – learning rate</p></li>
<li><p><strong>beta</strong> – factor for decreasing learning rate</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>matrix of vertex/word representation (n x d)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.denialConstraints">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">denialConstraints</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataFrame</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">constraintsFrame</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.denialConstraints" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function considers some constraints indicating statements that can NOT happen in the data (denial constraints).</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">EXAMPLE</span><span class="p">:</span>
<span class="n">dataFrame</span><span class="p">:</span>

     <span class="n">rank</span>       <span class="n">discipline</span>   <span class="n">yrs</span><span class="o">.</span><span class="n">since</span><span class="o">.</span><span class="n">phd</span>   <span class="n">yrs</span><span class="o">.</span><span class="n">service</span>   <span class="n">sex</span>      <span class="n">salary</span>
<span class="mi">1</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">19</span>              <span class="mi">18</span>            <span class="n">Male</span>     <span class="mi">139750</span>
<span class="mi">2</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">20</span>              <span class="mi">16</span>            <span class="n">Male</span>     <span class="mi">173200</span>
<span class="mi">3</span>    <span class="n">AsstProf</span>   <span class="n">B</span>            <span class="mi">3</span>               <span class="mi">3</span>             <span class="n">Male</span>     <span class="mf">79750.56</span>
<span class="mi">4</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">45</span>              <span class="mi">39</span>            <span class="n">Male</span>     <span class="mi">115000</span>
<span class="mi">5</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">40</span>              <span class="mi">40</span>            <span class="n">Male</span>     <span class="mi">141500</span>
<span class="mi">6</span>    <span class="n">AssocProf</span>  <span class="n">B</span>            <span class="mi">6</span>               <span class="mi">6</span>             <span class="n">Male</span>     <span class="mi">97000</span>
<span class="mi">7</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">30</span>              <span class="mi">23</span>            <span class="n">Male</span>     <span class="mi">175000</span>
<span class="mi">8</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">45</span>              <span class="mi">45</span>            <span class="n">Male</span>     <span class="mi">147765</span>
<span class="mi">9</span>    <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">21</span>              <span class="mi">20</span>            <span class="n">Male</span>     <span class="mi">119250</span>
<span class="mi">10</span>   <span class="n">Prof</span>       <span class="n">B</span>            <span class="mi">18</span>              <span class="mi">18</span>            <span class="n">Female</span>   <span class="mi">129000</span>
<span class="mi">11</span>   <span class="n">AssocProf</span>  <span class="n">B</span>            <span class="mi">12</span>              <span class="mi">8</span>             <span class="n">Male</span>     <span class="mi">119800</span>
<span class="mi">12</span>   <span class="n">AsstProf</span>   <span class="n">B</span>            <span class="mi">7</span>               <span class="mi">2</span>             <span class="n">Male</span>     <span class="mi">79800</span>
<span class="mi">13</span>   <span class="n">AsstProf</span>   <span class="n">B</span>            <span class="mi">1</span>               <span class="mi">1</span>             <span class="n">Male</span>     <span class="mi">77700</span>

<span class="n">constraintsFrame</span><span class="p">:</span>

<span class="n">idx</span>   <span class="n">constraint</span><span class="o">.</span><span class="n">type</span>   <span class="n">group</span><span class="o">.</span><span class="n">by</span>   <span class="n">group</span><span class="o">.</span><span class="n">variable</span>      <span class="n">group</span><span class="o">.</span><span class="n">option</span>   <span class="n">variable1</span>      <span class="n">relation</span>   <span class="n">variable2</span>
<span class="mi">1</span>     <span class="n">variableCompare</span>   <span class="n">FALSE</span>                                         <span class="n">yrs</span><span class="o">.</span><span class="n">since</span><span class="o">.</span><span class="n">phd</span>  <span class="o">&lt;</span>          <span class="n">yrs</span><span class="o">.</span><span class="n">service</span>
<span class="mi">2</span>     <span class="n">instanceCompare</span>   <span class="n">TRUE</span>       <span class="n">rank</span>                <span class="n">Prof</span>           <span class="n">yrs</span><span class="o">.</span><span class="n">service</span>    <span class="o">&gt;&lt;</span>         <span class="n">salary</span>
<span class="mi">3</span>     <span class="n">valueCompare</span>      <span class="n">FALSE</span>                                         <span class="n">salary</span>         <span class="o">=</span>          <span class="mi">78182</span>
<span class="mi">4</span>     <span class="n">variableCompare</span>   <span class="n">TRUE</span>       <span class="n">discipline</span>          <span class="n">B</span>              <span class="n">yrs</span><span class="o">.</span><span class="n">service</span>    <span class="o">&gt;</span>          <span class="n">yrs</span><span class="o">.</span><span class="n">since</span><span class="o">.</span><span class="n">phd</span>
</pre></div>
</div>
<p>Example: explanation of constraint 2 –&gt; it can’t happen that one professor of rank Prof has more years of service than other, but lower salary.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>dataFrame</strong> – frame which columns represent the variables of the data and the rows correspond
to different tuples or instances.
Recommended to have a column indexing the instances from 1 to N (N=number of instances).</p></li>
<li><p><strong>constraintsFrame</strong> – frame with fixed columns and each row representing one constraint.
1. idx: (double) index of the constraint, from 1 to M (number of constraints)
2. constraint.type: (string) The constraints can be of 3 different kinds:
- variableCompare: for each instance, it will compare the values of two variables (with a relation &lt;, &gt; or =).
- valueCompare: for each instance, it will compare a fixed value and a variable value (with a relation &lt;, &gt; or =).
- instanceCompare: for every couple of instances, it will compare the relation between two variables,
ie  if the value of the variable 1 in instance 1 is lower/higher than the value of variable 1 in instance 2,
then the value of of variable 2 in instance 2 can’t be lower/higher than the value of variable 2 in instance 2.
3. group.by: (boolean) if TRUE only one group of data (defined by a variable option) will be considered for the constraint.
4. group.variable: (string, only if group.by TRUE) name of the variable (column in dataFrame) that will divide our data in groups.
5. group.option: (only if group.by TRUE) option of the group.variable that defines the group to consider.
6. variable1: (string) first variable to compare (name of column in dataFrame).
7. relation: (string) can be &lt; , &gt; or = in the case of variableCompare and valueCompare, and &lt; &gt;, &lt; &lt; , &gt; &lt; or &gt; &gt;
in the case of instanceCompare
8. variable2: (string) second variable to compare (name of column in dataFrame) or fixed value for the case of valueCompare.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix of 2 columns.
- First column shows the indexes of dataFrame that are wrong.
- Second column shows the index of the denial constraint that is fulfilled
If there are no wrong instances to show (0 constrains fulfilled) –&gt; WrongInstances=matrix(0,1,2)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.differenceStatistics">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">differenceStatistics</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.differenceStatistics" title="Link to this definition"></a></dt>
<dd><p>Prints the difference statistics of two matrices given, to indicate how
they are different. This can be used for instance in comparison of lossy
compression techniques, that reduce the fidelity of the data.</p>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.discoverFD">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">discoverFD</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">threshold</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.discoverFD" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements builtin for finding functional dependencies</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input Matrix X, encoded Matrix if data is categorical</p></li>
<li><p><strong>Mask</strong> – A row vector for interested features i.e. Mask =[1, 0, 1]
will exclude the second column from processing</p></li>
<li><p><strong>threshold</strong> – threshold value in interval [0, 1] for robust FDs</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>matrix of functional dependencies</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.dist">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">dist</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.dist" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns Euclidean distance matrix (distances between N n-dimensional points)</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – Matrix to calculate the distance inside</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Euclidean distance matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.dmv">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">dmv</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.dmv" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The dmv-function is used to find disguised missing values utilising syntactical pattern recognition.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input Frame</p></li>
<li><p><strong>threshold</strong> – Threshold value in interval [0, 1] for dominant pattern per column (e.g., 0.8 means
that 80% of the entries per column must adhere this pattern to be dominant)</p></li>
<li><p><strong>replace</strong> – The string disguised missing values are replaced with</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Frame X including detected disguised missing values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.ema">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">ema</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">search_iterations</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">freq</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">alpha</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">beta</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">gamma</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.ema" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function imputes values with exponential moving average (single, double or triple).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Frame that contains time series data that needs to be imputed
search_iterations       Integer –      Budget iterations for parameter optimization,
used if parameters weren’t set</p></li>
<li><p><strong>mode</strong> – Type of EMA method. Either “single”, “double” or “triple”</p></li>
<li><p><strong>freq</strong> – Seasonality when using triple EMA.</p></li>
<li><p><strong>alpha</strong> – alpha- value for EMA</p></li>
<li><p><strong>beta</strong> – beta- value for EMA</p></li>
<li><p><strong>gamma</strong> – gamma- value for EMA</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Frame with EMA results</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.executePipeline">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">executePipeline</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">pipeline</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Xtrain</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Ytrain</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Xtest</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Ytest</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">metaList</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">hyperParameters</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">flagsCount</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.executePipeline" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function execute pipeline.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>logical</strong> – <p>—</p>
</p></li>
<li><p><strong>pipeline</strong> – <p>—</p>
</p></li>
<li><p><strong>X</strong> – <p>—</p>
</p></li>
<li><p><strong>Y</strong> – <p>—</p>
</p></li>
<li><p><strong>Xtest</strong> – <p>—</p>
</p></li>
<li><p><strong>Ytest</strong> – <p>—</p>
</p></li>
<li><p><strong>metaList</strong> – <p>—</p>
</p></li>
<li><p><strong>hyperParameters</strong> – <p>—</p>
</p></li>
<li><p><strong>hpForPruning</strong> – <p>—</p>
</p></li>
<li><p><strong>changesByOp</strong> – <p>—</p>
</p></li>
<li><p><strong>flagsCount</strong> – <p>—</p>
</p></li>
<li><p><strong>test</strong> – <p>—</p>
</p></li>
<li><p><strong>verbose</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p><p>—</p>
</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p><p>—</p>
</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p><p>—</p>
</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.ffPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">ffPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">model</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.ffPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function makes prediction given data and trained feedforward neural network model</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>Model</strong> – Trained ff neural network model</p></li>
<li><p><strong>X</strong> – Data used for making predictions</p></li>
<li><p><strong>batch_size</strong> – Batch size</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Predicted value</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.ffTrain">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">ffTrain</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_activation</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">loss_fcn</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.ffTrain" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function trains simple feed-forward neural network. The architecture of the
networks is: affine1 -&gt; relu -&gt; dropout -&gt; affine2 -&gt; configurable output activation function.
Hidden layer has 128 neurons. Dropout rate is 0.35. Input and output sizes are inferred from X and Y.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Training data</p></li>
<li><p><strong>Y</strong> – Labels/Target values</p></li>
<li><p><strong>batch_size</strong> – Batch size</p></li>
<li><p><strong>epochs</strong> – Number of epochs</p></li>
<li><p><strong>learning_rate</strong> – Learning rate</p></li>
<li><p><strong>out_activation</strong> – User specified output activation function. Possible values:
“sigmoid”, “relu”, “lrelu”, “tanh”, “softmax”, “logits” (no activation).</p></li>
<li><p><strong>loss_fcn</strong> – User specified loss function. Possible values:
“l1”, “l2”, “log_loss”, “logcosh_loss”, “cel” (cross-entropy loss).</p></li>
<li><p><strong>shuffle</strong> – Flag which indicates if dataset should be shuffled or not</p></li>
<li><p><strong>validation_split</strong> – Fraction of training set used as validation set</p></li>
<li><p><strong>seed</strong> – Seed for model initialization</p></li>
<li><p><strong>verbose</strong> – Flag which indicates if function should print to stdout</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Trained model which can be used in ffPredict</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.fit_pipeline">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">fit_pipeline</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">trainData</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">testData</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pip</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">applyFunc</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">hp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">evaluationFunc</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">evalFunHp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.fit_pipeline" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script will read the dirty and clean data, then it will apply the best pipeline on dirty data
and then will classify both cleaned dataset and check if the cleaned dataset is performing same as original dataset
in terms of classification accuracy</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>trainData</strong> – <p>—</p>
</p></li>
<li><p><strong>testData</strong> – <p>—</p>
</p></li>
<li><p><strong>metaData</strong> – <p>—</p>
</p></li>
<li><p><strong>lp</strong> – <p>—</p>
</p></li>
<li><p><strong>pip</strong> – <p>—</p>
</p></li>
<li><p><strong>hp</strong> – <p>—</p>
</p></li>
<li><p><strong>evaluationFunc</strong> – <p>—</p>
</p></li>
<li><p><strong>evalFunHp</strong> – <p>—</p>
</p></li>
<li><p><strong>isLastLabel</strong> – <p>—</p>
</p></li>
<li><p><strong>correctTypos</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.fixInvalidLengths">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">fixInvalidLengths</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">F1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.fixInvalidLengths" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Fix invalid lengths</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>F1</strong> – <p>—</p>
</p></li>
<li><p><strong>mask</strong> – <p>—</p>
</p></li>
<li><p><strong>ql</strong> – <p>—</p>
</p></li>
<li><p><strong>qu</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.fixInvalidLengthsApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">fixInvalidLengthsApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">qLow</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">qUp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.fixInvalidLengthsApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Fix invalid lengths</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – <p>—</p>
</p></li>
<li><p><strong>mask</strong> – <p>—</p>
</p></li>
<li><p><strong>ql</strong> – <p>—</p>
</p></li>
<li><p><strong>qu</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.flattenQuantile">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">flattenQuantile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">P</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.flattenQuantile" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the quantiles requested, but treating the input matrix X as a flattened matrix
to return quantiles of all cells as if it was a continuous allocation.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix with values to extract quantiles from.</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.frameSort">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">frameSort</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">F</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.frameSort" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Related to [SYSTEMDS-2662] dependency function for cleaning pipelines
Built-in for sorting frames</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>F</strong> – Data frame of string values</p></li>
<li><p><strong>mask</strong> – matrix for identifying string columns</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>sorted dataset by column 1 in decreasing order</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.frequencyEncode">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">frequencyEncode</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.frequencyEncode" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>function frequency conversion</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – dataset x</p></li>
<li><p><strong>mask</strong> – mask of the columns for frequency conversion</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>categorical columns are replaced with their frequencies</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>the frequency counts for the different categoricals</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.frequencyEncodeApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">frequencyEncodeApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">freqCount</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.frequencyEncodeApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>frequency code apply</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – dataset x</p></li>
<li><p><strong>freqCount</strong> – the frequency counts for the different categoricals</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>categorical columns are replaced with their frequencies given</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.garch">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">garch</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">kmax</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">momentum</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">start_stepsize</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">end_stepsize</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">start_vicinity</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">end_vicinity</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sim_seed</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.garch" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This is a builtin function that implements GARCH(1,1), a statistical model used in analyzing time-series data where the variance
error is believed to be serially autocorrelated</p>
<p>COMMENTS
This has some drawbacks: slow convergence of optimization (sort of simulated annealing/gradient descent)
TODO: use BFGS or BHHH if it is available (this are go to methods)
TODO: (only then) extend to garch(p,q); otherwise the search space is way too big for the current method</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – The input Matrix to apply Arima on.</p></li>
<li><p><strong>kmax</strong> – Number of iterations</p></li>
<li><p><strong>momentum</strong> – Momentum for momentum-gradient descent (set to 0 to deactivate)</p></li>
<li><p><strong>start_stepsize</strong> – Initial gradient-descent stepsize</p></li>
<li><p><strong>end_stepsize</strong> – gradient-descent stepsize at end (linear descent)</p></li>
<li><p><strong>start_vicinity</strong> – proportion of randomness of restart-location for gradient descent at beginning</p></li>
<li><p><strong>end_vicinity</strong> – same at end (linear decay)</p></li>
<li><p><strong>sim_seed</strong> – seed for simulation of process on fitted coefficients</p></li>
<li><p><strong>verbose</strong> – verbosity, comments during fitting</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>simulated garch(1,1) process on fitted coefficients</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>variances of simulated fitted process</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>onstant term of fitted process</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>1-st arch-coefficient of fitted process</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>1-st garch-coefficient of fitted process</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.gaussianClassifier">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">gaussianClassifier</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">D</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.gaussianClassifier" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Computes the parameters needed for Gaussian Classification.
Thus it computes the following per class: the prior probability,
the inverse covariance matrix, the mean per feature and the determinant
of the covariance matrix. Furthermore (if not explicitly defined), it
adds some small smoothing value along the variances, to prevent
numerical errors / instabilities.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>D</strong> – Input matrix (training set)</p></li>
<li><p><strong>C</strong> – Target vector</p></li>
<li><p><strong>varSmoothing</strong> – Smoothing factor for variances</p></li>
<li><p><strong>verbose</strong> – Print accuracy of the training set</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Vector storing the class prior probabilities</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix storing the means of the classes</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>List of inverse covariance matrices</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Vector storing the determinants of the classes</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.getAccuracy">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">getAccuracy</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">yhat</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.getAccuracy" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function compute the weighted and simple accuracy for given predictions</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>y</strong> – Ground truth (Actual Labels)</p></li>
<li><p><strong>yhat</strong> – Predictions (Predicted labels)</p></li>
<li><p><strong>isWeighted</strong> – Flag for weighted or non-weighted accuracy calculation</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>accuracy of the predicted labels</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.glm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">glm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.glm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script solves GLM regression using NEWTON/FISHER scoring with trust regions. The glm-function is a flexible
generalization of ordinary linear regression that allows for response variables that have error distribution models.</p>
<p>In addition, some GLM statistics are provided as console output by setting verbose=TRUE, one comma-separated name-value
pair per each line, as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">--------------------------------------------------------------------------------------------</span>
<span class="n">TERMINATION_CODE</span>      <span class="n">A</span> <span class="n">positive</span> <span class="n">integer</span> <span class="n">indicating</span> <span class="n">success</span><span class="o">/</span><span class="n">failure</span> <span class="k">as</span> <span class="n">follows</span><span class="p">:</span>
                      <span class="mi">1</span> <span class="o">=</span> <span class="n">Converged</span> <span class="n">successfully</span><span class="p">;</span> <span class="mi">2</span> <span class="o">=</span> <span class="n">Maximum</span> <span class="n">number</span> <span class="n">of</span> <span class="n">iterations</span> <span class="n">reached</span><span class="p">;</span> 
                      <span class="mi">3</span> <span class="o">=</span> <span class="n">Input</span> <span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y</span><span class="p">)</span> <span class="n">out</span> <span class="n">of</span> <span class="nb">range</span><span class="p">;</span> <span class="mi">4</span> <span class="o">=</span> <span class="n">Distribution</span><span class="o">/</span><span class="n">link</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">supported</span>
<span class="n">BETA_MIN</span>              <span class="n">Smallest</span> <span class="n">beta</span> <span class="n">value</span> <span class="p">(</span><span class="n">regression</span> <span class="n">coefficient</span><span class="p">),</span> <span class="n">excluding</span> <span class="n">the</span> <span class="n">intercept</span>
<span class="n">BETA_MIN_INDEX</span>        <span class="n">Column</span> <span class="n">index</span> <span class="k">for</span> <span class="n">the</span> <span class="n">smallest</span> <span class="n">beta</span> <span class="n">value</span>
<span class="n">BETA_MAX</span>              <span class="n">Largest</span> <span class="n">beta</span> <span class="n">value</span> <span class="p">(</span><span class="n">regression</span> <span class="n">coefficient</span><span class="p">),</span> <span class="n">excluding</span> <span class="n">the</span> <span class="n">intercept</span>
<span class="n">BETA_MAX_INDEX</span>        <span class="n">Column</span> <span class="n">index</span> <span class="k">for</span> <span class="n">the</span> <span class="n">largest</span> <span class="n">beta</span> <span class="n">value</span>
<span class="n">INTERCEPT</span>             <span class="n">Intercept</span> <span class="n">value</span><span class="p">,</span> <span class="ow">or</span> <span class="n">NaN</span> <span class="k">if</span> <span class="n">there</span> <span class="ow">is</span> <span class="n">no</span> <span class="n">intercept</span> <span class="p">(</span><span class="k">if</span> <span class="n">icpt</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">DISPERSION</span>            <span class="n">Dispersion</span> <span class="n">used</span> <span class="n">to</span> <span class="n">scale</span> <span class="n">deviance</span><span class="p">,</span> <span class="n">provided</span> <span class="k">as</span> <span class="s2">&quot;disp&quot;</span> <span class="nb">input</span> <span class="n">parameter</span>
                      <span class="ow">or</span> <span class="n">estimated</span> <span class="p">(</span><span class="n">same</span> <span class="k">as</span> <span class="n">DISPERSION_EST</span><span class="p">)</span> <span class="k">if</span> <span class="n">the</span> <span class="s2">&quot;disp&quot;</span> <span class="n">parameter</span> <span class="ow">is</span> <span class="o">&lt;=</span> <span class="mi">0</span>
<span class="n">DISPERSION_EST</span>        <span class="n">Dispersion</span> <span class="n">estimated</span> <span class="kn">from</span> <span class="nn">the</span> <span class="n">dataset</span>
<span class="n">DEVIANCE_UNSCALED</span>     <span class="n">Deviance</span> <span class="kn">from</span> <span class="nn">the</span> <span class="n">saturated</span> <span class="n">model</span><span class="p">,</span> <span class="n">assuming</span> <span class="n">dispersion</span> <span class="o">==</span> <span class="mf">1.0</span>
<span class="n">DEVIANCE_SCALED</span>       <span class="n">Deviance</span> <span class="kn">from</span> <span class="nn">the</span> <span class="n">saturated</span> <span class="n">model</span><span class="p">,</span> <span class="n">scaled</span> <span class="n">by</span> <span class="n">the</span> <span class="n">DISPERSION</span> <span class="n">value</span>
<span class="o">--------------------------------------------------------------------------------------------</span>

<span class="n">The</span> <span class="n">Log</span> <span class="n">file</span><span class="p">,</span> <span class="n">when</span> <span class="n">requested</span><span class="p">,</span> <span class="n">contains</span> <span class="n">the</span> <span class="n">following</span> <span class="n">per</span><span class="o">-</span><span class="n">iteration</span> <span class="n">variables</span> <span class="ow">in</span> <span class="n">CSV</span> <span class="nb">format</span><span class="p">,</span>
<span class="n">each</span> <span class="n">line</span> <span class="n">containing</span> <span class="n">triple</span> <span class="p">(</span><span class="n">NAME</span><span class="p">,</span> <span class="n">ITERATION</span><span class="p">,</span> <span class="n">VALUE</span><span class="p">)</span> <span class="k">with</span> <span class="n">ITERATION</span> <span class="o">=</span> <span class="mi">0</span> <span class="k">for</span> <span class="n">initial</span> <span class="n">values</span><span class="p">:</span>

<span class="o">--------------------------------------------------------------------------------------------</span>
<span class="n">NUM_CG_ITERS</span>          <span class="n">Number</span> <span class="n">of</span> <span class="n">inner</span> <span class="p">(</span><span class="n">Conj</span><span class="o">.</span><span class="n">Gradient</span><span class="p">)</span> <span class="n">iterations</span> <span class="ow">in</span> <span class="n">this</span> <span class="n">outer</span> <span class="n">iteration</span>
<span class="n">IS_TRUST_REACHED</span>      <span class="mi">1</span> <span class="o">=</span> <span class="n">trust</span> <span class="n">region</span> <span class="n">boundary</span> <span class="n">was</span> <span class="n">reached</span><span class="p">,</span> <span class="mi">0</span> <span class="o">=</span> <span class="n">otherwise</span>
<span class="n">POINT_STEP_NORM</span>       <span class="n">L2</span><span class="o">-</span><span class="n">norm</span> <span class="n">of</span> <span class="n">iteration</span> <span class="n">step</span> <span class="kn">from</span> <span class="nn">old</span> <span class="n">point</span> <span class="p">(</span><span class="n">i</span><span class="o">.</span><span class="n">e</span><span class="o">.</span> <span class="s2">&quot;beta&quot;</span><span class="p">)</span> <span class="n">to</span> <span class="n">new</span> <span class="n">point</span>
<span class="n">OBJECTIVE</span>             <span class="n">The</span> <span class="n">loss</span> <span class="n">function</span> <span class="n">we</span> <span class="n">minimize</span> <span class="p">(</span><span class="n">i</span><span class="o">.</span><span class="n">e</span><span class="o">.</span> <span class="n">negative</span> <span class="n">partial</span> <span class="n">log</span><span class="o">-</span><span class="n">likelihood</span><span class="p">)</span>
<span class="n">OBJ_DROP_REAL</span>         <span class="n">Reduction</span> <span class="ow">in</span> <span class="n">the</span> <span class="n">objective</span> <span class="n">during</span> <span class="n">this</span> <span class="n">iteration</span><span class="p">,</span> <span class="n">actual</span> <span class="n">value</span>
<span class="n">OBJ_DROP_PRED</span>         <span class="n">Reduction</span> <span class="ow">in</span> <span class="n">the</span> <span class="n">objective</span> <span class="n">predicted</span> <span class="n">by</span> <span class="n">a</span> <span class="n">quadratic</span> <span class="n">approximation</span>
<span class="n">OBJ_DROP_RATIO</span>        <span class="n">Actual</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">predicted</span> <span class="n">reduction</span> <span class="n">ratio</span><span class="p">,</span> <span class="n">used</span> <span class="n">to</span> <span class="n">update</span> <span class="n">the</span> <span class="n">trust</span> <span class="n">region</span>
<span class="n">GRADIENT_NORM</span>         <span class="n">L2</span><span class="o">-</span><span class="n">norm</span> <span class="n">of</span> <span class="n">the</span> <span class="n">loss</span> <span class="n">function</span> <span class="n">gradient</span> <span class="p">(</span><span class="n">NOTE</span><span class="p">:</span> <span class="n">sometimes</span> <span class="n">omitted</span><span class="p">)</span>
<span class="n">LINEAR_TERM_MIN</span>       <span class="n">The</span> <span class="n">minimum</span> <span class="n">value</span> <span class="n">of</span> <span class="n">X</span> <span class="o">%*%</span> <span class="n">beta</span><span class="p">,</span> <span class="n">used</span> <span class="n">to</span> <span class="n">check</span> <span class="k">for</span> <span class="n">overflows</span>
<span class="n">LINEAR_TERM_MAX</span>       <span class="n">The</span> <span class="n">maximum</span> <span class="n">value</span> <span class="n">of</span> <span class="n">X</span> <span class="o">%*%</span> <span class="n">beta</span><span class="p">,</span> <span class="n">used</span> <span class="n">to</span> <span class="n">check</span> <span class="k">for</span> <span class="n">overflows</span>
<span class="n">IS_POINT_UPDATED</span>      <span class="mi">1</span> <span class="o">=</span> <span class="n">new</span> <span class="n">point</span> <span class="n">accepted</span><span class="p">;</span> <span class="mi">0</span> <span class="o">=</span> <span class="n">new</span> <span class="n">point</span> <span class="n">rejected</span><span class="p">,</span> <span class="n">old</span> <span class="n">point</span> <span class="n">restored</span>
<span class="n">TRUST_DELTA</span>           <span class="n">Updated</span> <span class="n">trust</span> <span class="n">region</span> <span class="n">size</span><span class="p">,</span> <span class="n">the</span> <span class="s2">&quot;delta&quot;</span>
<span class="o">--------------------------------------------------------------------------------------------</span>
</pre></div>
</div>
<p>SOME OF THE SUPPORTED GLM DISTRIBUTION FAMILIES
AND LINK FUNCTIONS:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>dfam vpow link lpow  Distribution.link   nical?
---------------------------------------------------
 1   0.0   1  -1.0   Gaussian.inverse
 1   0.0   1   0.0   Gaussian.log
 1   0.0   1   1.0   Gaussian.id          Yes
 1   1.0   1   0.0   Poisson.log          Yes
 1   1.0   1   0.5   Poisson.sqrt
 1   1.0   1   1.0   Poisson.id
 1   2.0   1  -1.0   Gamma.inverse        Yes
 1   2.0   1   0.0   Gamma.log
 1   2.0   1   1.0   Gamma.id
 1   3.0   1  -2.0   InvGaussian.1/mu^2   Yes
 1   3.0   1  -1.0   InvGaussian.inverse
 1   3.0   1   0.0   InvGaussian.log
 1   3.0   1   1.0   InvGaussian.id
 1    *    1    *    AnyVariance.AnyLink
---------------------------------------------------
 2    *    1   0.0   Binomial.log
 2    *    1   0.5   Binomial.sqrt
 2    *    2    *    Binomial.logit       Yes
 2    *    3    *    Binomial.probit
 2    *    4    *    Binomial.cloglog
 2    *    5    *    Binomial.cauchit
---------------------------------------------------
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – matrix X of feature vectors</p></li>
<li><p><strong>Y</strong> – matrix Y with either 1 or 2 columns:
if dfam = 2, Y is 1-column Bernoulli or 2-column Binomial (#pos, #neg)</p></li>
<li><p><strong>dfam</strong> – Distribution family code: 1 = Power, 2 = Binomial</p></li>
<li><p><strong>vpow</strong> – Power for Variance defined as (mean)^power (ignored if dfam != 1):
0.0 = Gaussian, 1.0 = Poisson, 2.0 = Gamma, 3.0 = Inverse Gaussian</p></li>
<li><p><strong>link</strong> – Link function code: 0 = canonical (depends on distribution),
1 = Power, 2 = Logit, 3 = Probit, 4 = Cloglog, 5 = Cauchit</p></li>
<li><p><strong>lpow</strong> – Power for Link function defined as (mean)^power (ignored if link != 1):
-2.0 = 1/mu^2, -1.0 = reciprocal, 0.0 = log, 0.5 = sqrt, 1.0 = identity</p></li>
<li><p><strong>yneg</strong> – Response value for Bernoulli “No” label, usually 0.0 or -1.0</p></li>
<li><p><strong>icpt</strong> – Intercept presence, X columns shifting and rescaling:
0 = no intercept, no shifting, no rescaling;
1 = add intercept, but neither shift nor rescale X;
2 = add intercept, shift &amp; rescale X columns to mean = 0, variance = 1</p></li>
<li><p><strong>reg</strong> – Regularization parameter (lambda) for L2 regularization</p></li>
<li><p><strong>tol</strong> – Tolerance (epsilon)</p></li>
<li><p><strong>disp</strong> – (Over-)dispersion value, or 0.0 to estimate it from data</p></li>
<li><p><strong>moi</strong> – Maximum number of outer (Newton / Fisher Scoring) iterations</p></li>
<li><p><strong>mii</strong> – Maximum number of inner (Conjugate Gradient) iterations, 0 = no maximum</p></li>
<li><p><strong>verbose</strong> – if the Algorithm should be verbose</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix beta, whose size depends on icpt:
icpt=0: ncol(X) x 1;  icpt=1: (ncol(X) + 1) x 1;  icpt=2: (ncol(X) + 1) x 2</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.glmPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">glmPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">B</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.glmPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Applies the estimated parameters of a GLM type regression to a new dataset</p>
<p>Additional statistics are printed one per each line, in the following</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">CSV</span> <span class="nb">format</span><span class="p">:</span> <span class="n">NAME</span><span class="p">,[</span><span class="n">COLUMN</span><span class="p">],[</span><span class="n">SCALED</span><span class="p">],</span><span class="n">VALUE</span>
<span class="o">---</span>
<span class="n">NAME</span>   <span class="ow">is</span> <span class="n">the</span> <span class="n">string</span> <span class="n">identifier</span> <span class="k">for</span> <span class="n">the</span> <span class="n">statistic</span><span class="p">,</span> <span class="n">see</span> <span class="n">the</span> <span class="n">table</span> <span class="n">below</span><span class="o">.</span>
<span class="n">COLUMN</span> <span class="ow">is</span> <span class="n">an</span> <span class="n">optional</span> <span class="n">integer</span> <span class="n">value</span> <span class="n">that</span> <span class="n">specifies</span> <span class="n">the</span> <span class="n">Y</span><span class="o">-</span><span class="n">column</span> <span class="k">for</span> <span class="n">per</span><span class="o">-</span><span class="n">column</span> <span class="n">statistics</span><span class="p">;</span>
       <span class="n">note</span> <span class="n">that</span> <span class="n">a</span> <span class="n">Binomial</span><span class="o">/</span><span class="n">Multinomial</span> <span class="n">one</span><span class="o">-</span><span class="n">column</span> <span class="n">Y</span> <span class="nb">input</span> <span class="ow">is</span> <span class="n">converted</span> <span class="n">into</span> <span class="n">multi</span><span class="o">-</span><span class="n">column</span><span class="o">.</span>
<span class="n">SCALED</span> <span class="ow">is</span> <span class="n">an</span> <span class="n">optional</span> <span class="n">Boolean</span> <span class="n">value</span> <span class="p">(</span><span class="n">TRUE</span> <span class="ow">or</span> <span class="n">FALSE</span><span class="p">)</span> <span class="n">that</span> <span class="n">tells</span> <span class="n">us</span> <span class="n">whether</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">the</span> <span class="nb">input</span>
         <span class="n">dispersion</span> <span class="n">parameter</span> <span class="p">(</span><span class="n">disp</span><span class="p">)</span> <span class="n">scaling</span> <span class="n">has</span> <span class="n">been</span> <span class="n">applied</span> <span class="n">to</span> <span class="n">this</span> <span class="n">statistic</span><span class="o">.</span>
<span class="n">VALUE</span>  <span class="ow">is</span> <span class="n">the</span> <span class="n">value</span> <span class="n">of</span> <span class="n">the</span> <span class="n">statistic</span><span class="o">.</span>
<span class="o">---</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">NAME</span>                  <span class="n">COLUMN</span>  <span class="n">SCALED</span>  <span class="n">MEANING</span>
<span class="o">---------------------------------------------------------------------------------------------</span>
<span class="n">LOGLHOOD_Z</span>                      <span class="o">+</span>     <span class="n">Log</span><span class="o">-</span><span class="n">Likelihood</span> <span class="n">Z</span><span class="o">-</span><span class="n">score</span> <span class="p">(</span><span class="ow">in</span> <span class="n">st</span><span class="o">.</span><span class="n">dev</span><span class="s1">&#39;s from mean)</span>
<span class="n">LOGLHOOD_Z_PVAL</span>                 <span class="o">+</span>     <span class="n">Log</span><span class="o">-</span><span class="n">Likelihood</span> <span class="n">Z</span><span class="o">-</span><span class="n">score</span> <span class="n">p</span><span class="o">-</span><span class="n">value</span>
<span class="n">PEARSON_X2</span>                      <span class="o">+</span>     <span class="n">Pearson</span> <span class="n">residual</span> <span class="n">X</span><span class="o">^</span><span class="mi">2</span> <span class="n">statistic</span>
<span class="n">PEARSON_X2_BY_DF</span>                <span class="o">+</span>     <span class="n">Pearson</span> <span class="n">X</span><span class="o">^</span><span class="mi">2</span> <span class="n">divided</span> <span class="n">by</span> <span class="n">degrees</span> <span class="n">of</span> <span class="n">freedom</span>
<span class="n">PEARSON_X2_PVAL</span>                 <span class="o">+</span>     <span class="n">Pearson</span> <span class="n">X</span><span class="o">^</span><span class="mi">2</span> <span class="n">p</span><span class="o">-</span><span class="n">value</span>
<span class="n">DEVIANCE_G2</span>                     <span class="o">+</span>     <span class="n">Deviance</span> <span class="kn">from</span> <span class="nn">saturated</span> <span class="n">model</span> <span class="n">G</span><span class="o">^</span><span class="mi">2</span> <span class="n">statistic</span>
<span class="n">DEVIANCE_G2_BY_DF</span>               <span class="o">+</span>     <span class="n">Deviance</span> <span class="n">G</span><span class="o">^</span><span class="mi">2</span> <span class="n">divided</span> <span class="n">by</span> <span class="n">degrees</span> <span class="n">of</span> <span class="n">freedom</span>
<span class="n">DEVIANCE_G2_PVAL</span>                <span class="o">+</span>     <span class="n">Deviance</span> <span class="n">G</span><span class="o">^</span><span class="mi">2</span> <span class="n">p</span><span class="o">-</span><span class="n">value</span>
<span class="n">AVG_TOT_Y</span>               <span class="o">+</span>             <span class="n">Average</span> <span class="n">of</span> <span class="n">Y</span> <span class="n">column</span> <span class="k">for</span> <span class="n">a</span> <span class="n">single</span> <span class="n">response</span> <span class="n">value</span>
<span class="n">STDEV_TOT_Y</span>             <span class="o">+</span>             <span class="n">St</span><span class="o">.</span><span class="n">Dev</span><span class="o">.</span> <span class="n">of</span> <span class="n">Y</span> <span class="n">column</span> <span class="k">for</span> <span class="n">a</span> <span class="n">single</span> <span class="n">response</span> <span class="n">value</span>
<span class="n">AVG_RES_Y</span>               <span class="o">+</span>             <span class="n">Average</span> <span class="n">of</span> <span class="n">column</span> <span class="n">residual</span><span class="p">,</span> <span class="n">i</span><span class="o">.</span><span class="n">e</span><span class="o">.</span> <span class="n">of</span> <span class="n">Y</span> <span class="o">-</span> <span class="n">mean</span><span class="p">(</span><span class="n">Y</span><span class="o">|</span><span class="n">X</span><span class="p">)</span>
<span class="n">STDEV_RES_Y</span>             <span class="o">+</span>             <span class="n">St</span><span class="o">.</span><span class="n">Dev</span><span class="o">.</span> <span class="n">of</span> <span class="n">column</span> <span class="n">residual</span><span class="p">,</span> <span class="n">i</span><span class="o">.</span><span class="n">e</span><span class="o">.</span> <span class="n">of</span> <span class="n">Y</span> <span class="o">-</span> <span class="n">mean</span><span class="p">(</span><span class="n">Y</span><span class="o">|</span><span class="n">X</span><span class="p">)</span>
<span class="n">PRED_STDEV_RES</span>          <span class="o">+</span>       <span class="o">+</span>     <span class="n">Model</span><span class="o">-</span><span class="n">predicted</span> <span class="n">St</span><span class="o">.</span><span class="n">Dev</span><span class="o">.</span> <span class="n">of</span> <span class="n">column</span> <span class="n">residual</span>
<span class="n">R2</span>                      <span class="o">+</span>             <span class="n">R</span><span class="o">^</span><span class="mi">2</span> <span class="n">of</span> <span class="n">Y</span> <span class="n">column</span> <span class="n">residual</span> <span class="k">with</span> <span class="n">bias</span> <span class="n">included</span>
<span class="n">ADJUSTED_R2</span>             <span class="o">+</span>             <span class="n">Adjusted</span> <span class="n">R</span><span class="o">^</span><span class="mi">2</span> <span class="n">of</span> <span class="n">Y</span> <span class="n">column</span> <span class="n">residual</span> <span class="k">with</span> <span class="n">bias</span> <span class="n">included</span>
<span class="n">R2_NOBIAS</span>               <span class="o">+</span>             <span class="n">R</span><span class="o">^</span><span class="mi">2</span> <span class="n">of</span> <span class="n">Y</span> <span class="n">column</span> <span class="n">residual</span> <span class="k">with</span> <span class="n">bias</span> <span class="n">subtracted</span>
<span class="n">ADJUSTED_R2_NOBIAS</span>      <span class="o">+</span>             <span class="n">Adjusted</span> <span class="n">R</span><span class="o">^</span><span class="mi">2</span> <span class="n">of</span> <span class="n">Y</span> <span class="n">column</span> <span class="n">residual</span> <span class="k">with</span> <span class="n">bias</span> <span class="n">subtracted</span>
<span class="o">---------------------------------------------------------------------------------------------</span>
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X of records (feature vectors)</p></li>
<li><p><strong>B</strong> – GLM regression parameters (the betas), with dimensions
ncol(X)   x k: do not add intercept
ncol(X)+1 x k: add intercept as given by the last B-row
if k &gt; 1, use only B[, 1] unless it is Multinomial Logit (dfam=3)</p></li>
<li><p><strong>ytest</strong> – Response matrix Y, with the following dimensions:
nrow(X) x 1  : for all distributions (dfam=1 or 2 or 3)
nrow(X) x 2  : for Binomial (dfam=2) given by (#pos, #neg) counts
nrow(X) x k+1: for Multinomial (dfam=3) given by category counts</p></li>
<li><p><strong>dfam</strong> – GLM distribution family: 1 = Power, 2 = Binomial, 3 = Multinomial Logit</p></li>
<li><p><strong>vpow</strong> – Power for Variance defined as (mean)^power (ignored if dfam != 1):
0.0 = Gaussian, 1.0 = Poisson, 2.0 = Gamma, 3.0 = Inverse Gaussian</p></li>
<li><p><strong>link</strong> – Link function code: 0 = canonical (depends on distribution), 1 = Power,
2 = Logit, 3 = Probit, 4 = Cloglog, 5 = Cauchit; ignored if Multinomial</p></li>
<li><p><strong>lpow</strong> – Power for Link function defined as (mean)^power (ignored if link != 1):
-2.0 = 1/mu^2, -1.0 = reciprocal, 0.0 = log, 0.5 = sqrt, 1.0 = identity</p></li>
<li><p><strong>disp</strong> – Dispersion value, when available</p></li>
<li><p><strong>verbose</strong> – Print statistics to stdout</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix M of predicted means/probabilities:
nrow(X) x 1  : for Power-type distributions (dfam=1)
nrow(X) x 2  : for Binomial distribution (dfam=2), column 2 is “No”
nrow(X) x k+1: for Multinomial Logit (dfam=3), col# k+1 is baseline</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.gmm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">gmm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.gmm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Gaussian Mixture Model training algorithm.
There are four different types of covariance matrices
i.e., VVV, EEE, VVI, VII and two initialization methods namely “kmeans” and “random”.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Dataset input to fit the GMM model</p></li>
<li><p><strong>n_components</strong> – Number of components to use in the Gaussian mixture model</p></li>
<li><p><strong>model</strong> – “VVV”: unequal variance (full),each component has its own general covariance matrix
“EEE”: equal variance (tied), all components share the same general covariance matrix
“VVI”: spherical, unequal volume (diag), each component has its own diagonal
covariance matrix
“VII”: spherical, equal volume (spherical), each component has its own single variance</p></li>
<li><p><strong>init_param</strong> – Initialization algorithm to use to initialize the gaussian weights, valid inputs are:
“kmeans” or “random”</p></li>
<li><p><strong>iterations</strong> – Number of iterations</p></li>
<li><p><strong>reg_covar</strong> – Regularization parameter for covariance matrix</p></li>
<li><p><strong>tol</strong> – Tolerance value for convergence</p></li>
<li><p><strong>seed</strong> – The seed value to initialize the values for fitting the GMM.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The predictions made by the gaussian model on the X input dataset</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Probability of the predictions given the X input dataset</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Number of estimated parameters</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Bayesian information criterion for best iteration</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Fitted clusters mean</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Fitted precision matrix for each mixture</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The weight matrix:
A matrix whose [i,k]th entry is the probability
that observation i in the test data belongs to the kth class</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.gmmPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">gmmPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mu</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">precisions_cholesky</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.gmmPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Prediction function for a Gaussian Mixture Model (gmm).
Compute posterior probabilities for new instances given the variance and mean of fitted dat.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Dataset input to predict the labels from</p></li>
<li><p><strong>weight</strong> – Weight of learned model:
A matrix whose [i,k]th entry is the probability
that observation i in the test data belongs to the kth class</p></li>
<li><p><strong>mu</strong> – Fitted clusters mean</p></li>
<li><p><strong>precisions_cholesky</strong> – Fitted precision matrix for each mixture</p></li>
<li><p><strong>model</strong> – “VVV”: unequal variance (full),each component has its own general covariance matrix
“EEE”: equal variance (tied), all components share the same general covariance matrix
“VVI”: spherical, unequal volume (diag), each component has its own diagonal
covariance matrix
“VII”: spherical, equal volume (spherical), each component has its own single variance</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The predictions made by the gaussian model on the X input dataset</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Probability of the predictions given the X input dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.gnmf">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">gnmf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">rnk</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.gnmf" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The gnmf-function does Gaussian Non-Negative Matrix Factorization. In this, a matrix X is factorized into two
matrices W and H, such that all three matrices have no negative elements. This non-negativity makes the resulting
matrices easier to inspect.</p>
<p>References:
[Chao Liu, Hung-chih Yang, Jinliang Fan, Li-Wei He, Yi-Min Wang:
Distributed nonnegative matrix factorization for web-scale dyadic
data analysis on mapreduce. WWW 2010: 681-690]</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors.</p></li>
<li><p><strong>rnk</strong> – Number of components into which matrix X is to be factored</p></li>
<li><p><strong>eps</strong> – Tolerance</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>List of pattern matrices, one for each repetition</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>List of amplitude matrices, one for each repetition</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.gridSearch">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">gridSearch</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">predict</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">params</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">paramValues</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.gridSearch" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The gridSearch-function is used to find the optimal hyper-parameters of a model which results in the most
accurate predictions. This function takes train and eval functions by name.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>y</strong> – Input Matrix of vectors.</p></li>
<li><p><strong>train</strong> – Name ft of the train function to call via ft(trainArgs)</p></li>
<li><p><strong>predict</strong> – Name fp of the loss function to call via fp((predictArgs,B))</p></li>
<li><p><strong>numB</strong> – Maximum number of parameters in model B (pass the max because the size
may vary with parameters like icpt or multi-class classification)</p></li>
<li><p><strong>params</strong> – List of varied hyper-parameter names</p></li>
<li><p><strong>dataArgs</strong> – List of data parameters (to identify data parameters by name i.e. list(“X”, “Y”))</p></li>
<li><p><strong>paramValues</strong> – List of matrices providing the parameter values as
columnvectors for position-aligned hyper-parameters in ‘params’</p></li>
<li><p><strong>trainArgs</strong> – named List of arguments to pass to the ‘train’ function, where
gridSearch replaces enumerated hyper-parameter by name, if
not provided or an empty list, the lm parameters are used</p></li>
<li><p><strong>predictArgs</strong> – List of arguments to pass to the ‘predict’ function, where
gridSearch appends the trained models at the end, if
not provided or an empty list, list(X, y) is used instead</p></li>
<li><p><strong>cv</strong> – flag enabling k-fold cross validation, otherwise training loss</p></li>
<li><p><strong>cvk</strong> – if cv=TRUE, specifies the the number of folds, otherwise ignored</p></li>
<li><p><strong>verbose</strong> – flag for verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix[Double]the trained model with minimal loss (by the ‘predict’ function)
Multi-column models are returned as a column-major linearized column vector</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>one-row frame w/ optimal hyper-parameters (by ‘params’ position)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.hospitalResidencyMatch">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">hospitalResidencyMatch</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">R</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">H</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">capacity</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.hospitalResidencyMatch" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script computes a solution for the hospital residency match problem.</p>
<p>Residents.mtx:
2.0,1.0,3.0
1.0,2.0,3.0
1.0,2.0,0.0</p>
<p>Since it is an ORDERED  matrix, this means that Resident 1 (row 1) likes hospital 2 the most, followed by hospital 1 and hospital 3.
If it was UNORDERED, this would mean that resident 1 (row 1) likes hospital 3 the most (since the value at [1,3] is the row max),
followed by hospital 1 (2.0 preference value) and hospital 2 (1.0 preference value).</p>
<p>Hospitals.mtx:
2.0,1.0,0.0
0.0,1.0,2.0
1.0,2.0,0.0</p>
<p>Since it is an UNORDERED matrix this means that Hospital 1 (row 1) likes Resident 1 the most (since the value at [1,1] is the row max).</p>
<p>capacity.mtx
1.0
1.0
1.0</p>
<p>residencyMatch.mtx
2.0,0.0,0.0
1.0,0.0,0.0
0.0,2.0,0.0</p>
<p>hospitalMatch.mtx
0.0,1.0,0.0
0.0,0.0,2.0
1.0,0.0,0.0</p>
<p>Resident 1 has matched with Hospital 3 (since [1,3] is non-zero) at a preference level of 2.0.
Resident 2 has matched with Hospital 1 (since [2,1] is non-zero) at a preference level of 1.0.
Resident 3 has matched with Hospital 2 (since [3,2] is non-zero) at a preference level of 2.0.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>R</strong> – Residents matrix R.
It must be an ORDERED  matrix.</p></li>
<li><p><strong>H</strong> – Hospitals matrix H.
It must be an UNORDRED matrix.</p></li>
<li><p><strong>capacity</strong> – capacity of Hospitals matrix C.
It must be a [n*1] matrix with non zero values.
i.e. the leftmost value in a row is the most preferred partner’s index.
i.e. the leftmost value in a row in P is the preference value for the acceptor
with index 1 and vice-versa (higher is better).</p></li>
<li><p><strong>verbose</strong> – If the operation is verbose</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Result Matrix
If cell [i,j] is non-zero, it means that Resident i has matched with Hospital j.
Further, if cell [i,j] is non-zero, it holds the preference value that led to the match.</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Result Matrix
If cell [i,j] is non-zero, it means that Resident i has matched with Hospital j.
Further, if cell [i,j] is non-zero, it holds the preference value that led to the match.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.hyperband">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">hyperband</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X_train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y_train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">X_val</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y_val</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">params</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">paramRanges</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.hyperband" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The hyperband-function is used for hyper parameter optimization and is based on multi-armed bandits and early
elimination. Through multiple parallel brackets and consecutive trials it will return the hyper parameter combination
which performed best on a validation dataset. A set of hyper parameter combinations is drawn from uniform distributions
with given ranges; Those make up the candidates for hyperband. Notes:
hyperband is hard-coded for lmCG, and uses lmPredict for validation
hyperband is hard-coded to use the number of iterations as a resource
hyperband can only optimize continuous hyperparameters</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X_train</strong> – Input Matrix of training vectors</p></li>
<li><p><strong>y_train</strong> – Labels for training vectors</p></li>
<li><p><strong>X_val</strong> – Input Matrix of validation vectors</p></li>
<li><p><strong>y_val</strong> – Labels for validation vectors</p></li>
<li><p><strong>params</strong> – List of parameters to optimize</p></li>
<li><p><strong>paramRanges</strong> – The min and max values for the uniform distributions to draw from.
One row per hyper parameter, first column specifies min, second column max value.</p></li>
<li><p><strong>R</strong> – Controls number of candidates evaluated</p></li>
<li><p><strong>eta</strong> – Determines fraction of candidates to keep after each trial</p></li>
<li><p><strong>verbose</strong> – If TRUE print messages are activated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>1-column matrix of weights of best performing candidate</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>hyper parameters of best performing candidate</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_brightness">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_brightness</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">channel_max</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_brightness" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The img_brightness-function is an image data augmentation function. It changes the brightness of the image.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input matrix/image</p></li>
<li><p><strong>value</strong> – The amount of brightness to be changed for the image</p></li>
<li><p><strong>channel_max</strong> – Maximum value of the brightness of the image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output matrix/image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_brightness_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_brightness_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">channel_max</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_brightness_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The img_brightness_linearized-function is an image data augmentation function. It changes the brightness of one or multiple images.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input matrix/image (can represent multiple images every row of the matrix represents a linearized image)</p></li>
<li><p><strong>value</strong> – The amount of brightness to be changed for the image</p></li>
<li><p><strong>channel_max</strong> – Maximum value of the brightness of the image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output matrix/images  (every row of the matrix represents a linearized image)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_crop">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_crop</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">x_offset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y_offset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_crop" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The img_crop-function is an image data augmentation function. It cuts out a subregion of an image.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input matrix/image</p></li>
<li><p><strong>w</strong> – The width of the subregion required</p></li>
<li><p><strong>h</strong> – The height of the subregion required</p></li>
<li><p><strong>x_offset</strong> – The horizontal coordinate in the image to begin the crop operation</p></li>
<li><p><strong>y_offset</strong> – The vertical coordinate in the image to begin the crop operation</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Cropped matrix/image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_crop_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_crop_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">x_offset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y_offset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_crop_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The img_crop_linearized cuts out a rectangular section of multiple linearized images.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Linearized input images as 2D matrix</p></li>
<li><p><strong>w</strong> – The width of the subregion required</p></li>
<li><p><strong>h</strong> – The height of the subregion required</p></li>
<li><p><strong>x_offset</strong> – The horizontal offset for the center of the crop region</p></li>
<li><p><strong>y_offset</strong> – The vertical offset for the center of the crop region</p></li>
<li><p><strong>s_cols</strong> – Width of a single image</p></li>
<li><p><strong>s_rows</strong> – Height of a single image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Cropped images as linearized 2D matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_cutout">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_cutout</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">width</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">height</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_cutout" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Image Cutout function replaces a rectangular section of an image with a constant value.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>x</strong> – Column index of the top left corner of the rectangle (starting at 1)</p></li>
<li><p><strong>y</strong> – Row index of the top left corner of the rectangle (starting at 1)</p></li>
<li><p><strong>width</strong> – Width of the rectangle (must be positive)</p></li>
<li><p><strong>height</strong> – Height of the rectangle (must be positive)</p></li>
<li><p><strong>fill_value</strong> – The value to set for the rectangle</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_cutout_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_cutout_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">width</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">height</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_cutout_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Image Cutout function replaces a rectangular section of an image with a constant value.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input images as linearized 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>x</strong> – Column index of the top left corner of the rectangle (starting at 1)</p></li>
<li><p><strong>y</strong> – Row index of the top left corner of the rectangle (starting at 1)</p></li>
<li><p><strong>width</strong> – Width of the rectangle (must be positive)</p></li>
<li><p><strong>height</strong> – Height of the rectangle (must be positive)</p></li>
<li><p><strong>fill_value</strong> – The value to set for the rectangle</p></li>
<li><p><strong>s_cols</strong> – Width of a single image</p></li>
<li><p><strong>s_rows</strong> – Height of a single image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output images as linearized 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_invert">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_invert</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_invert" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This is an image data augmentation function. It inverts an image.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image</p></li>
<li><p><strong>max_value</strong> – The maximum value pixels can have</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_invert_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_invert_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_invert_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This is an image data augmentation function. It inverts an image.It can handle one or multiple images</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input matrix/image (every row of the matrix represents a linearized image)</p></li>
<li><p><strong>max_value</strong> – The maximum value pixels can have</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output images (every row of the matrix represents a linearized image)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_mirror">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_mirror</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">horizontal_axis</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_mirror" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function is an image data augmentation function.
It flips an image on the X (horizontal) or Y (vertical) axis.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input matrix/image</p></li>
<li><p><strong>max_value</strong> – The maximum value pixels can have</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Flipped matrix/image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_mirror_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_mirror_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_matrix</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">horizontal_axis</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">original_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">original_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_mirror_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function has  the same functionality with img_mirror but it handles multiple images at
the same time. Each row of the input and output matrix represents a linearized image/matrix
It flips an image on the X (horizontal) or Y (vertical) axis.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_matrix</strong> – Input matrix/image (every row represents a linearized matrix/image)</p></li>
<li><p><strong>horizontal_axis</strong> – flip either in X or Y axis</p></li>
<li><p><strong>original_rows</strong> – number of rows in the original 2-D images</p></li>
<li><p><strong>original_cols</strong> – number of cols in the original 2-D images</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output matrix/image  (every row represents a linearized matrix/image)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_posterize">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_posterize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">bits</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_posterize" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Image Posterize function limits pixel values to 2^bits different values in the range [0, 255].
Assumes the input image can attain values in the range [0, 255].</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image</p></li>
<li><p><strong>bits</strong> – The number of bits keep for the values.
1 means black and white, 8 means every integer between 0 and 255.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_posterize_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_posterize_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">bits</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_posterize_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Linearized Image Posterize function limits pixel values to 2^bits different values in the range [0, 255].
Assumes the input image can attain values in the range [0, 255].</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Row linearized input images as 2D matrix</p></li>
<li><p><strong>bits</strong> – The number of bits keep for the values.
1 means black and white, 8 means every integer between 0 and 255.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Row linearized output images as 2D matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_rotate">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_rotate</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">radians</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_rotate" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Image Rotate function rotates the input image counter-clockwise around the center.
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>radians</strong> – The value by which to rotate in radian.</p></li>
<li><p><strong>fill_value</strong> – The background color revealed by the rotation</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_rotate_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_rotate_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">radians</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_rotate_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Linearized Image Rotate function rotates the linearized input images counter-clockwise around the center.
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Linearized input images as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>radians</strong> – The value by which to rotate in radian.</p></li>
<li><p><strong>fill_value</strong> – The background color revealed by the rotation</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output images in linearized form as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_sample_pairing">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_sample_pairing</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">img_in2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_sample_pairing" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The image sample pairing function blends two images together.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in1</strong> – First input image</p></li>
<li><p><strong>img_in2</strong> – Second input image</p></li>
<li><p><strong>weight</strong> – The weight given to the second image.
0 means only img_in1, 1 means only img_in2 will be visible</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_sample_pairing_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_sample_pairing_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">img_in2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">weight</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_sample_pairing_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The image sample pairing function blends two images together.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in1</strong> – input matrix/image (every row is a linearized image)</p></li>
<li><p><strong>img_in2</strong> – Second input image (one image represented as a single row linearized matrix)</p></li>
<li><p><strong>weight</strong> – The weight given to the second image.
0 means only img_in1, 1 means only img_in2 will be visible</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_shear">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_shear</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">shear_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shear_y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_shear" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function applies a shearing transformation to an image.
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>shear_x</strong> – Shearing factor for horizontal shearing</p></li>
<li><p><strong>shear_y</strong> – Shearing factor for vertical shearing</p></li>
<li><p><strong>fill_value</strong> – The background color revealed by the shearing</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_shear_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_shear_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">shear_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">shear_y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_shear_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function applies a shearing transformation to linearized images.
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Linearized input images as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>shear_x</strong> – Shearing factor for horizontal shearing</p></li>
<li><p><strong>shear_y</strong> – Shearing factor for vertical shearing</p></li>
<li><p><strong>fill_value</strong> – The background color revealed by the shearing</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output images in linearized form as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_transform">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_transform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">b</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">c</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">d</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">e</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">f</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_transform" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Image Transform function applies an affine transformation to an image.
Optionally resizes the image (without scaling).
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>out_w</strong> – Width of the output image</p></li>
<li><p><strong>out_h</strong> – Height of the output image</p></li>
<li><p><strong>a</strong><strong>,</strong><strong>b</strong><strong>,</strong><strong>c</strong><strong>,</strong><strong>d</strong><strong>,</strong><strong>e</strong><strong>,</strong><strong>f</strong> – The first two rows of the affine matrix in row-major order</p></li>
<li><p><strong>fill_value</strong> – The background of the image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_transform_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_transform_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">b</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">c</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">d</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">e</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">f</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">s_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_transform_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Linearized Image Transform function applies an affine transformation to linearized images.
Optionally resizes the image (without scaling).
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Linearized input images as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>out_w</strong> – Width of the output matrix</p></li>
<li><p><strong>out_h</strong> – Height of the output matrix</p></li>
<li><p><strong>a</strong><strong>,</strong><strong>b</strong><strong>,</strong><strong>c</strong><strong>,</strong><strong>d</strong><strong>,</strong><strong>e</strong><strong>,</strong><strong>f</strong> – The first two rows of the affine matrix in row-major order</p></li>
<li><p><strong>fill_value</strong> – The background of an image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output images in linearized form as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_translate">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_translate</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">offset_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">offset_y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_translate" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Image Translate function translates the image.
Optionally resizes the image (without scaling).
Uses nearest neighbor sampling.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input image as 2D matrix with top left corner at [1, 1]</p></li>
<li><p><strong>offset_x</strong> – The distance to move the image in x direction</p></li>
<li><p><strong>offset_y</strong> – The distance to move the image in y direction</p></li>
<li><p><strong>out_w</strong> – Width of the output image</p></li>
<li><p><strong>out_h</strong> – Height of the output image</p></li>
<li><p><strong>fill_value</strong> – The background of the image</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output image as 2D matrix with top left corner at [1, 1]</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.img_translate_linearized">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">img_translate_linearized</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">img_in</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">offset_x</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">offset_y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">out_h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fill_value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">o_w</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">o_h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.img_translate_linearized" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function has  the same functionality with img_translate but it handles multiple images at
the same time. Each row of the input and output matrix represents a linearized image/matrix
It translates the image and Optionally resizes the image (without scaling).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>img_in</strong> – Input matrix/image (every row represents a linearized matrix/image)</p></li>
<li><p><strong>offset_x</strong> – The distance to move the image in x direction</p></li>
<li><p><strong>offset_y</strong> – The distance to move the image in y direction</p></li>
<li><p><strong>out_w</strong> – Width of the output image</p></li>
<li><p><strong>out_h</strong> – Height of the output image</p></li>
<li><p><strong>fill_value</strong> – The background of the image</p></li>
<li><p><strong>o_w</strong> – Width of the original 2D images</p></li>
<li><p><strong>o_h</strong> – Height of the original 2D images</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output matrix/image  (every row represents a linearized matrix/image)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.impurityMeasures">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">impurityMeasures</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">R</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">method</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.impurityMeasures" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function computes the measure of impurity for the given dataset based on the passed method (gini or entropy).
The current version expects the target vector to contain only 0 or 1 values.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix.</p></li>
<li><p><strong>Y</strong> – Target vector containing 0 and 1 values.</p></li>
<li><p><strong>R</strong> – Vector indicating whether a feature is categorical or continuous.
1 denotes a continuous feature, 2 denotes a categorical feature.</p></li>
<li><p><strong>n_bins</strong> – Number of bins for binning in case of scale features.</p></li>
<li><p><strong>method</strong> – String indicating the method to use; either “entropy” or “gini”.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>(1 x ncol(X)) row vector containing information/gini gain for
each feature of the dataset.
In case of gini, the values denote the gini gains, i.e. how much
impurity was removed with the respective split. The higher the
value, the better the split.
In case of entropy, the values denote the information gain, i.e.
how much entropy was removed. The higher the information gain,
the better the split.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByFD">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByFD</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">threshold</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByFD" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements builtin for imputing missing values from observed values (if exist) using robust functional dependencies</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Vector X, source attribute of functional dependency</p></li>
<li><p><strong>Y</strong> – Vector Y, target attribute of functional dependency and imputation</p></li>
<li><p><strong>threshold</strong> – threshold value in interval [0, 1] for robust FDs</p></li>
<li><p><strong>verbose</strong> – flag for printing verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Vector Y, with missing values mapped to a new max value</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Vector Y, with imputed missing values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByFDApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByFDApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y_imp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByFDApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements builtin for imputing missing values from observed values (if exist) using robust functional dependencies</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>source</strong> – source attribute to use for imputation and error correction</p></li>
<li><p><strong>target</strong> – attribute to be fixed</p></li>
<li><p><strong>threshold</strong> – threshold value in interval [0, 1] for robust FDs</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix with possible imputations</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByMean">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByMean</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByMean" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>impute the data by mean value and if the feature is categorical then by mode value
Related to [SYSTEMDS-2662] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>mask</strong> – A 0/1 row vector for identifying numeric (0) and categorical features (1)</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByMeanApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByMeanApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">imputedVec</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByMeanApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>impute the data by mean value and if the feature is categorical then by mode value
Related to [SYSTEMDS-2662] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>imputationVector</strong> – column mean vector</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByMedian">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByMedian</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByMedian" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Related to [SYSTEMDS-2662] dependency function for cleaning pipelines</p>
<p>impute the data by median value and if the feature is categorical then by mode value</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>mask</strong> – A 0/1 row vector for identifying numeric (0) and categorical features (1)</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByMedianApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByMedianApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">imputedVec</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByMedianApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>impute the data by median value and if the feature is categorical then by mode value
Related to [SYSTEMDS-2662] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>imputationVector</strong> – column median vector</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByMode">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByMode</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByMode" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function impute the data by mode value
Related to [SYSTEMDS-2902] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.imputeByModeApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">imputeByModeApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">imputedVec</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.imputeByModeApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>impute the data by most frequent value (recoded data only)
Related to [SYSTEMDS-2662] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>imputationVector</strong> – column mean vector</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.intersect">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">intersect</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.intersect" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements set intersection for numeric data</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – matrix X, set A</p></li>
<li><p><strong>Y</strong> – matrix Y, set B</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>intersection matrix, set of intersecting items</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.km">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">km</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">TE</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">GI</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">SI</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.km" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that implements the analysis of survival data with KAPLAN-MEIER estimates</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input matrix X containing the survival data:
timestamps, whether event occurred (1) or data is censored (0), and a
number of factors (categorical features) for grouping and/or stratifying</p></li>
<li><p><strong>TE</strong> – Column indices of X which contain timestamps (first entry) and event
information (second entry)</p></li>
<li><p><strong>GI</strong> – Column indices of X corresponding to the factors to be used for grouping</p></li>
<li><p><strong>SI</strong> – Column indices of X corresponding to the factors to be used for stratifying</p></li>
<li><p><strong>alpha</strong> – Parameter to compute 100*(1-alpha)% confidence intervals for the survivor
function and its median</p></li>
<li><p><strong>err_type</strong> – “greenwood” Parameter to specify the error type according to “greenwood” (the default) or “peto”</p></li>
<li><p><strong>conf_type</strong> – Parameter to modify the confidence interval; “plain” keeps the lower and
upper bound of the confidence interval unmodified, “log” (the default)
corresponds to logistic transformation and “log-log” corresponds to the
complementary log-log transformation</p></li>
<li><p><strong>test_type</strong> – If survival data for multiple groups is available specifies which test to
perform for comparing survival data across multiple groups: “none” (the default)
“log-rank” or “wilcoxon” test</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix KM whose dimension depends on the number of groups (denoted by g) and
strata (denoted by s) in the data:
each collection of 7 consecutive columns in KM corresponds to a unique
combination of groups and strata in the data with the following schema
1. col: timestamp
2. col: no. at risk
3. col: no. of events
4. col: Kaplan-Meier estimate of survivor function surv
5. col: standard error of surv
6. col: lower 100*(1-alpha)% confidence interval for surv
7. col: upper 100*(1-alpha)% confidence interval for surv</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix M whose dimension depends on the number of groups (g) and strata (s) in
the data (k denotes the number of factors used for grouping  ,i.e., ncol(GI) and
l denotes the number of factors used for stratifying, i.e., ncol(SI))
M[,1:k]: unique combination of values in the k factors used for grouping
M[,(k+1):(k+l)]: unique combination of values in the l factors used for stratifying
M[,k+l+1]: total number of records
M[,k+l+2]: total number of events
M[,k+l+3]: median of surv
M[,k+l+4]: lower 100*(1-alpha)% confidence interval of the median of surv
M[,k+l+5]: upper 100*(1-alpha)% confidence interval of the median of surv
If the number of groups and strata is equal to 1, M will have 4 columns with
M[,1]: total number of events
M[,2]: median of surv
M[,3]: lower 100*(1-alpha)% confidence interval of the median of surv
M[,4]: upper 100*(1-alpha)% confidence interval of the median of surv</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>If survival data from multiple groups available and ttype=log-rank or wilcoxon,
a 1 x 4 matrix T and an g x 5 matrix T_GROUPS_OE with
T_GROUPS_OE[,1] = no. of events
T_GROUPS_OE[,2] = observed value (O)
T_GROUPS_OE[,3] = expected value (E)
T_GROUPS_OE[,4] = (O-E)^2/E
T_GROUPS_OE[,5] = (O-E)^2/V
T[1,1] = no. of groups
T[1,2] = degree of freedom for Chi-squared distributed test statistic
T[1,3] = test statistic
T[1,4] = P-value</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.kmeans">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">kmeans</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.kmeans" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that implements the k-Means clustering algorithm</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – The input Matrix to do KMeans on.</p></li>
<li><p><strong>k</strong> – Number of centroids</p></li>
<li><p><strong>runs</strong> – Number of runs (with different initial centroids)</p></li>
<li><p><strong>max_iter</strong> – Maximum number of iterations per run</p></li>
<li><p><strong>eps</strong> – Tolerance (epsilon) for WCSS change ratio</p></li>
<li><p><strong>is_verbose</strong> – do not print per-iteration stats</p></li>
<li><p><strong>avg_sample_size_per_centroid</strong> – Average number of records per centroid in data samples</p></li>
<li><p><strong>seed</strong> – The seed used for initial sampling. If set to -1
random seeds are selected.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The mapping of records to centroids</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>The output matrix with the centroids</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.kmeansPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">kmeansPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.kmeansPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that does predictions based on a set of centroids provided.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – The input Matrix to do KMeans on.</p></li>
<li><p><strong>C</strong> – The input Centroids to map X onto.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The mapping of records to centroids</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.knn">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">knn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">Train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Test</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">CL</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">START_SELECTED</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.knn" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script implements KNN (K Nearest Neighbor) algorithm.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>Train</strong> – The input matrix as features</p></li>
<li><p><strong>Test</strong> – The input matrix for nearest neighbor search</p></li>
<li><p><strong>CL</strong> – The input matrix as target</p></li>
<li><p><strong>CL_T</strong> – The target type of matrix CL whether
columns in CL are continuous ( =1 ) or
categorical ( =2 ) or not specified ( =0 )</p></li>
<li><p><strong>trans_continuous</strong> – Option flag for continuous feature transformed to [-1,1]:
FALSE = do not transform continuous variable;
TRUE = transform continuous variable;</p></li>
<li><p><strong>k_value</strong> – k value for KNN, ignore if select_k enable</p></li>
<li><p><strong>select_k</strong> – Use k selection algorithm to estimate k (TRUE means yes)</p></li>
<li><p><strong>k_min</strong> – Min k value(  available if select_k = 1 )</p></li>
<li><p><strong>k_max</strong> – Max k value(  available if select_k = 1 )</p></li>
<li><p><strong>select_feature</strong> – Use feature selection algorithm to select feature (TRUE means yes)</p></li>
<li><p><strong>feature_max</strong> – Max feature selection</p></li>
<li><p><strong>interval</strong> – Interval value for K selecting (  available if select_k = 1 )</p></li>
<li><p><strong>feature_importance</strong> – Use feature importance algorithm to estimate each feature
(TRUE means yes)</p></li>
<li><p><strong>predict_con_tg</strong> – Continuous  target predict function: mean(=0) or median(=1)</p></li>
<li><p><strong>START_SELECTED</strong> – feature selection initial value</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Applied clusters to X</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Cluster matrix</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Feature importance value</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.knnGraph">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">knnGraph</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.knnGraph" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin for k nearest neighbor graph construction</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – <p>—</p>
</p></li>
<li><p><strong>k</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.knnbf">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">knnbf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">T</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.knnbf" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script implements KNN (K Nearest Neighbor) algorithm.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – <p>—</p>
</p></li>
<li><p><strong>T</strong> – <p>—</p>
</p></li>
<li><p><strong>k_value</strong> – <p>—</p>
</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.l2svm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">l2svm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.l2svm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builting function implements binary-class Support Vector Machine (SVM)
with squared slack variables (l2 regularization).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix X (shape: m x n)</p></li>
<li><p><strong>Y</strong> – Label vector y of class labels (shape: m x 1), assumed binary
in -1/+1 or 1/2 encoding.</p></li>
<li><p><strong>intercept</strong> – Indicator if a bias column should be added to X and the model</p></li>
<li><p><strong>epsilon</strong> – Tolerance for early termination if the reduction of objective
function is less than epsilon times the initial objective</p></li>
<li><p><strong>reg</strong> – Regularization parameter (lambda) for L2 regularization</p></li>
<li><p><strong>maxIterations</strong> – Maximum number of conjugate gradient (outer) iterations</p></li>
<li><p><strong>maxii</strong> – Maximum number of line search (inner) iterations</p></li>
<li><p><strong>verbose</strong> – Indicator if training details should be printed</p></li>
<li><p><strong>columnId</strong> – An optional class ID used in verbose print output,
eg. used when L2SVM is used in MSVM.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Trained model/weights (shape: n x 1, w/ intercept: n+1)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.l2svmPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">l2svmPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">W</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.l2svmPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function Implements binary-class SVM with squared slack variables.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – matrix X of feature vectors to classify</p></li>
<li><p><strong>W</strong> – matrix of the trained variables</p></li>
<li><p><strong>verbose</strong> – Set to true if one wants print statements.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Classification Labels Raw, meaning not modified to clean
labels of 1’s and -1’s</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Classification Labels Maxed to ones and zeros.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lasso">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lasso</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lasso" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for the SpaRSA algorithm to perform lasso regression
(SpaRSA .. Sparse Reconstruction by Separable Approximation)</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – input feature matrix</p></li>
<li><p><strong>y</strong> – matrix Y columns of the design matrix</p></li>
<li><p><strong>tol</strong> – target convergence tolerance</p></li>
<li><p><strong>M</strong> – history length</p></li>
<li><p><strong>tau</strong> – regularization component</p></li>
<li><p><strong>maxi</strong> – maximum number of iterations until convergence</p></li>
<li><p><strong>verbose</strong> – if the builtin should be verbose</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>model matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lenetPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lenetPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">model</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Hin</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Win</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lenetPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function makes prediction given data and trained LeNet model</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>model</strong> – Trained LeNet model</p></li>
<li><p><strong>X</strong> – Input data matrix, of shape (N, C*Hin*Win)</p></li>
<li><p><strong>C</strong> – Number of input channels</p></li>
<li><p><strong>Hin</strong> – Input height</p></li>
<li><p><strong>Win</strong> – Input width</p></li>
<li><p><strong>batch_size</strong> – Batch size</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Predicted values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lenetTrain">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lenetTrain</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">X_val</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y_val</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Hin</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">Win</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lenetTrain" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function trains LeNet CNN. The architecture of the
networks is:conv1 -&gt; relu1 -&gt; pool1 -&gt; conv2 -&gt; relu2 -&gt; pool2 -&gt;
affine3 -&gt; relu3 -&gt; affine4 -&gt; softmax</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input data matrix, of shape (N, C*Hin*Win)</p></li>
<li><p><strong>Y</strong> – Target matrix, of shape (N, K)</p></li>
<li><p><strong>X_val</strong> – Validation data matrix, of shape (N, C*Hin*Win)</p></li>
<li><p><strong>Y_val</strong> – Validation target matrix, of shape (N, K)</p></li>
<li><p><strong>C</strong> – Number of input channels (dimensionality of input depth)</p></li>
<li><p><strong>Hin</strong> – Input width</p></li>
<li><p><strong>Win</strong> – Input height</p></li>
<li><p><strong>batch_size</strong> – Batch size</p></li>
<li><p><strong>epochs</strong> – Number of epochs</p></li>
<li><p><strong>lr</strong> – Learning rate</p></li>
<li><p><strong>mu</strong> – Momentum value</p></li>
<li><p><strong>decay</strong> – Learning rate decay</p></li>
<li><p><strong>reg</strong> – Regularization strength</p></li>
<li><p><strong>seed</strong> – Seed for model initialization</p></li>
<li><p><strong>verbose</strong> – Flag indicates if function should print to stdout</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Trained model which can be used in lenetPredict</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The lm-function solves linear regression using either the direct solve
method or the conjugate gradient algorithm depending on the input size
of the matrices (See lmDS-function and lmCG-function respectively).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors.</p></li>
<li><p><strong>y</strong> – 1-column matrix of response values.</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling the columns of X</p></li>
<li><p><strong>reg</strong> – Regularization constant (lambda) for L2-regularization. set to nonzero
for highly dependant/sparse/numerous features</p></li>
<li><p><strong>tol</strong> – Tolerance (epsilon); conjugate gradient procedure terminates early if L2
norm of the beta-residual is less than tolerance * its initial norm</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations. 0 = no maximum</p></li>
<li><p><strong>verbose</strong> – If TRUE print messages are activated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The model fit beta that can be used as input in lmPredict</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lmCG">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lmCG</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lmCG" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The lmCG function solves linear regression using the conjugate gradient algorithm</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors.</p></li>
<li><p><strong>y</strong> – 1-column matrix of response values.</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling the columns of X</p></li>
<li><p><strong>reg</strong> – Regularization constant (lambda) for L2-regularization. set to nonzero
for highly dependant/sparse/numerous features</p></li>
<li><p><strong>tol</strong> – Tolerance (epsilon) conjugate gradient procedure terminates early if L2
norm of the beta-residual is less than tolerance * its initial norm</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations. 0 = no maximum</p></li>
<li><p><strong>verbose</strong> – If TRUE print messages are activated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The model fit beta that can be used as input in lmPredict</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lmDS">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lmDS</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lmDS" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The lmDC function solves linear regression using the direct solve method</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors.</p></li>
<li><p><strong>y</strong> – 1-column matrix of response values.</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling the columns of X</p></li>
<li><p><strong>reg</strong> – Regularization constant (lambda) for L2-regularization. set to nonzero
for highly dependant/sparse/numerous features</p></li>
<li><p><strong>tol</strong> – Tolerance (epsilon) conjugate gradient procedure terminates early if L2
norm of the beta-residual is less than tolerance * its initial norm</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations. 0 = no maximum</p></li>
<li><p><strong>verbose</strong> – If TRUE print messages are activated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The model fit beta that can be used as input in lmPredict</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lmPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lmPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">B</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lmPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The lmPredict-function predicts the class of a feature vector</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors</p></li>
<li><p><strong>B</strong> – 1-column matrix of weights.</p></li>
<li><p><strong>ytest</strong> – test labels, used only for verbose output. can be set to matrix(0,1,1)
if verbose output is not wanted</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling the columns of X</p></li>
<li><p><strong>verbose</strong> – If TRUE print messages are activated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>1-column matrix of classes</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.lmPredictStats">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">lmPredictStats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">yhat</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ytest</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">lm</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.lmPredictStats" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function computes and prints a summary of accuracy
measures for regression problems.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>yhat</strong> – A column vector of predicted response values y</p></li>
<li><p><strong>ytest</strong> – A column vector of actual response values y</p></li>
<li><p><strong>lm</strong> – An indicator if used for linear regression model</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>A column vector holding avg_res, ss_avg_res, and R2</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.logSumExp">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">logSumExp</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">M</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.logSumExp" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Built-in LOGSUMEXP</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>M</strong> – matrix to perform Log sum exp on.</p></li>
<li><p><strong>margin</strong> – if the logsumexp of rows is required set margin = “row”
if the logsumexp of columns is required set margin = “col”
if set to “none” then a single scalar is returned computing logsumexp of matrix</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>a 1*1 matrix, row vector or column vector depends on margin value</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.mae">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">mae</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.mae" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the means absolute error between the two inputs</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Mean absolute error</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.mape">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">mape</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.mape" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the means absolute percentage error between the two inputs</p>
<p>Monash Time Series Forecasting Archive
Rakshitha Godahewaa,∗, Christoph Bergmeira , Geoffrey I. Webba , Rob J. Hyndmanb ,
Pablo Montero-Mansoc</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Mean absolute percentage error</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.matrixProfile">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">matrixProfile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">ts</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.matrixProfile" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that computes the MatrixProfile of a time series efficiently
using the SCRIMP++ algorithm.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">References</span><span class="p">:</span>
<span class="n">Yan</span> <span class="n">Zhu</span> <span class="n">et</span> <span class="n">al</span><span class="o">..</span> <span class="mf">2018.</span>
  <span class="n">Matrix</span> <span class="n">Profile</span> <span class="n">XI</span><span class="p">:</span> <span class="n">SCRIMP</span><span class="o">++</span><span class="p">:</span> <span class="n">Time</span> <span class="n">Series</span> <span class="n">Motif</span> <span class="n">Discovery</span> <span class="n">at</span> <span class="n">Interactive</span> <span class="n">Speeds</span><span class="o">.</span>
  <span class="mi">2018</span> <span class="n">IEEE</span> <span class="n">International</span> <span class="n">Conference</span> <span class="n">on</span> <span class="n">Data</span> <span class="n">Mining</span> <span class="p">(</span><span class="n">ICDM</span><span class="p">),</span> <span class="mi">2018</span><span class="p">,</span> <span class="n">pp</span><span class="o">.</span> <span class="mi">837</span><span class="o">-</span><span class="mf">846.</span>
  <span class="n">DOI</span><span class="p">:</span> <span class="mf">10.1109</span><span class="o">/</span><span class="n">ICDM</span><span class="mf">.2018.00099</span><span class="o">.</span>
  <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">cs</span><span class="o">.</span><span class="n">ucr</span><span class="o">.</span><span class="n">edu</span><span class="o">/~</span><span class="n">eamonn</span><span class="o">/</span><span class="n">SCRIMP_ICDM_camera_ready_updated</span><span class="o">.</span><span class="n">pdf</span>
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>ts</strong> – Time series to profile</p></li>
<li><p><strong>window_size</strong> – Sliding window size</p></li>
<li><p><strong>sample_percent</strong> – Degree of approximation
between zero and one (1
computes the exact solution)</p></li>
<li><p><strong>is_verbose</strong> – Print debug information</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The computed matrix profile</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Indices of least distances</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.mcc">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">mcc</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">predictions</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">labels</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.mcc" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Built-in function mcc: Matthews’ Correlation Coefficient for binary classification evaluation</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>predictions</strong> – Vector of predicted 0/1 values.
(requires setting ‘labels’ parameter)</p></li>
<li><p><strong>labels</strong> – Vector of 0/1 labels.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matthews’ Correlation Coefficient</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.mdedup">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">mdedup</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">LHSfeatures</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">LHSthreshold</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">RHSfeatures</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">RHSthreshold</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.mdedup" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Implements builtin for deduplication using matching dependencies (e.g. Street 0.95, City 0.90 -&gt; ZIP 1.0)
and Jaccard distance.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input Frame X</p></li>
<li><p><strong>LHSfeatures</strong> – A matrix 1xd with numbers of columns for MDs
(e.g. Street 0.95, City 0.90 -&gt; ZIP 1.0)</p></li>
<li><p><strong>LHSthreshold</strong> – A matrix 1xd with threshold values in interval [0, 1] for MDs</p></li>
<li><p><strong>RHSfeatures</strong> – A matrix 1xd with numbers of columns for MDs</p></li>
<li><p><strong>RHSthreshold</strong> – A matrix 1xd with threshold values in interval [0, 1] for MDs</p></li>
<li><p><strong>verbose</strong> – To print the output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix nx1 of duplicates</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.mice">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">mice</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cMask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.mice" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This Builtin function implements multiple imputation using Chained Equations (MICE)</p>
<p>Assumption missing value are represented with empty string i.e “,,” in CSV file  
variables with suffix n are storing continuos/numeric data and variables with 
suffix c are storing categorical data</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>cMask</strong> – A 0/1 row vector for identifying numeric (0) and categorical features (1)</p></li>
<li><p><strong>iter</strong> – Number of iteration for multiple imputations</p></li>
<li><p><strong>threshold</strong> – confidence value [0, 1] for robust imputation, values will only be imputed
if the predicted value has probability greater than threshold,
only applicable for categorical data</p></li>
<li><p><strong>verbose</strong> – Boolean value.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.miceApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">miceApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">meta</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">threshold</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">dM</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">betaList</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/list.html#systemds.operator.List" title="systemds.operator.nodes.list.List"><span class="pre">List</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.miceApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This Builtin function implements multiple imputation using Chained Equations (MICE)</p>
<p>Assumption missing value are represented with empty string i.e “,,” in CSV file  
variables with suffix n are storing continuos/numeric data and variables with 
suffix c are storing categorical data</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (Recoded Matrix for categorical features)</p></li>
<li><p><strong>mtea</strong> – A meta matrix with each rows storing values 1) mask of original matrix,
2) information of columns with missing values on  original data 0 for no missing value in column and 1 otherwise
3) dist values in each columns in original data 1 for continuous columns and colMax for categorical</p></li>
<li><p><strong>threshold</strong> – confidence value [0, 1] for robust imputation, values will only be imputed
if the predicted value has probability greater than threshold,
only applicable for categorical data</p></li>
<li><p><strong>dM</strong> – meta frame from OHE on original data</p></li>
<li><p><strong>betaList</strong> – List of machine learning models trained for each column imputation</p></li>
<li><p><strong>verbose</strong> – Boolean value.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>imputed dataset</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.mse">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">mse</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.mse" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the means square error between the two inputs</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Mean Square error</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.msmape">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">msmape</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.msmape" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the modified symmetric means absolute percentage error between the two inputs</p>
<p>Monash Time Series Forecasting Archive
Rakshitha Godahewaa,∗, Christoph Bergmeira , Geoffrey I. Webba , Rob J. Hyndmanb ,
Pablo Montero-Mansoc</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The modified symmetric mean absolute percentage error</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.msvm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">msvm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.msvm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function implements a multi-class Support Vector Machine (SVM)
with squared slack variables. The trained model comprises #classes
one-against-the-rest binary-class l2svm classification models.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix X (shape: m x n)</p></li>
<li><p><strong>Y</strong> – Label vector y of class labels (shape: m x 1),
where max(Y) is assumed to be the number of classes</p></li>
<li><p><strong>intercept</strong> – Indicator if a bias column should be added to X and the model</p></li>
<li><p><strong>epsilon</strong> – Tolerance for early termination if the reduction of objective
function is less than epsilon times the initial objective</p></li>
<li><p><strong>reg</strong> – Regularization parameter (lambda) for L2 regularization</p></li>
<li><p><strong>maxIterations</strong> – Maximum number of conjugate gradient (outer l2svm) iterations</p></li>
<li><p><strong>verbose</strong> – Indicator if training details should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Trained model/weights (shape: n x max(Y), w/ intercept: n+1)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.msvmPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">msvmPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">W</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.msvmPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This Scripts helps in applying an trained MSVM</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – matrix X of feature vectors to classify</p></li>
<li><p><strong>W</strong> – matrix of the trained variables</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Classification Labels Raw, meaning not modified to clean
Labeles of 1’s and -1’s</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Classification Labels Maxed to ones and zeros.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.multiLogReg">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">multiLogReg</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.multiLogReg" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Solves Multinomial Logistic Regression using Trust Region method.
(See: Trust Region Newton Method for Logistic Regression, Lin, Weng and Keerthi, JMLR 9 (2008) 627-650)
The largest label represents the baseline category; if label -1 or 0 is present, then it is
the baseline label (and it is converted to the largest label).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location to read the matrix of feature vectors</p></li>
<li><p><strong>Y</strong> – Location to read the matrix with category labels</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling X columns: 0 = no intercept,
no shifting, no rescaling; 1 = add intercept, but neither shift nor
rescale X; 2 = add intercept, shift &amp; rescale X columns to mean = 0, variance = 1</p></li>
<li><p><strong>tol</strong> – tolerance (“epsilon”)</p></li>
<li><p><strong>reg</strong> – regularization parameter (lambda = 1/C); intercept is not regularized</p></li>
<li><p><strong>maxi</strong> – max. number of outer (Newton) iterations</p></li>
<li><p><strong>maxii</strong> – max. number of inner (conjugate gradient) iterations, 0 = no max</p></li>
<li><p><strong>verbose</strong> – flag specifying if logging information should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>regression betas as output for prediction</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.multiLogRegPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">multiLogRegPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">B</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.multiLogRegPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>THIS SCRIPT APPLIES THE ESTIMATED PARAMETERS OF MULTINOMIAL LOGISTIC REGRESSION TO A NEW (TEST) DATASET
Matrix M of predicted means/probabilities, some statistics in CSV format (see below)</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix X</p></li>
<li><p><strong>B</strong> – Regression parameters betas</p></li>
<li><p><strong>Y</strong> – Response vector Y</p></li>
<li><p><strong>verbose</strong> – flag specifying if logging information should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix M of predicted means/probabilities</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Predicted response vector</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>scalar value of accuracy</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.na_locf">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">na_locf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.na_locf" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for imputing missing values using forward fill and backward fill techniques</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>option</strong> – String “locf” (last observation moved forward) to do forward fill
“nocb” (next observation carried backward) to do backward fill</p></li>
<li><p><strong>verbose</strong> – to print output on screen</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix with no missing values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.naiveBayes">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">naiveBayes</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">D</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.naiveBayes" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The naiveBayes-function computes the class conditional probabilities and class priors.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>D</strong> – One dimensional column matrix with N rows.</p></li>
<li><p><strong>C</strong> – One dimensional column matrix with N rows.</p></li>
<li><p><strong>laplace</strong> – Any Double value.</p></li>
<li><p><strong>verbose</strong> – Boolean value.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Class priors, One dimensional column matrix with N rows.</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Class conditional probabilities, One dimensional column matrix with N rows.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.naiveBayesPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">naiveBayesPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">P</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">C</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.naiveBayesPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The naiveBaysePredict-function predicts the scoring with a naive Bayes model.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of test data with N rows.</p></li>
<li><p><strong>P</strong> – Class priors, One dimensional column matrix with N rows.</p></li>
<li><p><strong>C</strong> – Class conditional probabilities, matrix with N rows</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>A matrix containing the top-K item-ids with highest predicted ratings.</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>A matrix containing predicted ratings.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.normalize">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">normalize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.normalize" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Min-max normalization (a.k.a. min-max scaling) to range [0,1]. For matrices 
of positive values, this normalization preserves the input sparsity.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – Input feature matrix of shape n-by-m</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Modified output feature matrix of shape n-by-m</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Column minima of shape 1-by-m</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Column maxima of shape 1-by-m</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.normalizeApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">normalizeApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cmin</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cmax</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.normalizeApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Min-max normalization (a.k.a. min-max scaling) to range [0,1], given 
existing min-max ranges. For matrices of positive values, this normalization 
preserves the input sparsity. The validity of the provided min-max range
and post-processing is under control of the caller.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix of shape n-by-m</p></li>
<li><p><strong>cmin</strong> – Column min of shape 1-by-m</p></li>
<li><p><strong>cmax</strong> – Column max of shape 1-by-m</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Modified output feature matrix of shape n-by-m</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.nrmse">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">nrmse</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.nrmse" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the normalized root means square error between the two inputs</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The normalized root means square error</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.outlier">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">outlier</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">opposite</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.outlier" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This outlier-function takes a matrix data set as input from where it determines
which point(s) have the largest difference from mean.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of Recoded dataset for outlier evaluation</p></li>
<li><p><strong>opposite</strong> – (1)TRUE for evaluating outlier from upper quartile range,
(0)FALSE for evaluating outlier from lower quartile range</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>matrix indicating outlier values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.outlierByArima">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">outlierByArima</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.outlierByArima" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Built-in function for detecting and repairing outliers in time series, by training an ARIMA model
and classifying values that are more than k standard-deviations away from the predicated values as outliers.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>k</strong> – threshold values 1, 2, 3 for 68%, 95%, 99.7% respectively (3-sigma rule)</p></li>
<li><p><strong>repairMethod</strong> – values: 0 = delete rows having outliers, 1 = replace outliers as zeros
2 = replace outliers as missing values</p></li>
<li><p><strong>p</strong> – non-seasonal AR order</p></li>
<li><p><strong>d</strong> – non-seasonal differencing order</p></li>
<li><p><strong>q</strong> – non-seasonal MA order</p></li>
<li><p><strong>P</strong> – seasonal AR order</p></li>
<li><p><strong>D</strong> – seasonal differencing order</p></li>
<li><p><strong>Q</strong> – seasonal MA order</p></li>
<li><p><strong>s</strong> – period in terms of number of time-steps</p></li>
<li><p><strong>include_mean</strong> – If the mean should be included</p></li>
<li><p><strong>solver</strong> – solver, is either “cg” or “jacobi”</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix X with no outliers</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.outlierByIQR">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">outlierByIQR</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_iterations</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.outlierByIQR" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for detecting and repairing outliers using standard deviation</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>k</strong> – a constant used to discern outliers k*IQR</p></li>
<li><p><strong>isIterative</strong> – iterative repair or single repair</p></li>
<li><p><strong>repairMethod</strong> – values: 0 = delete rows having outliers,
1 = replace outliers with zeros
2 = replace outliers as missing values</p></li>
<li><p><strong>max_iterations</strong> – values: 0 = arbitrary number of iteraition until all outliers are removed,
n = any constant defined by user</p></li>
<li><p><strong>verbose</strong> – flag specifying if logging information should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix X with no outliers</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.outlierByIQRApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">outlierByIQRApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Q1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Q3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">IQR</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">repairMethod</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.outlierByIQRApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for repairing outliers by IQR</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>Q1</strong> – first quartile</p></li>
<li><p><strong>Q3</strong> – third quartile</p></li>
<li><p><strong>IQR</strong> – Inter-quartile range</p></li>
<li><p><strong>k</strong> – a constant used to discern outliers k*IQR</p></li>
<li><p><strong>repairMethod</strong> – values: 0 = delete rows having outliers,
1 = replace outliers with zeros
2 = replace outliers as missing values</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix X with no outliers</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.outlierBySd">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">outlierBySd</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_iterations</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.outlierBySd" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for detecting and repairing outliers using standard deviation</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>k</strong> – threshold values 1, 2, 3 for 68%, 95%, 99.7% respectively (3-sigma rule)</p></li>
<li><p><strong>repairMethod</strong> – values: 0 = delete rows having outliers, 1 = replace outliers as  zeros
2 = replace outliers as missing values</p></li>
<li><p><strong>max_iterations</strong> – values: 0 = arbitrary number of iteration until all outliers are removed,
n = any constant defined by user</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix X with no outliers</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.outlierBySdApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">outlierBySdApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">colMean</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">colSD</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">repairMethod</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.outlierBySdApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for detecting and repairing outliers using standard deviation</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X</p></li>
<li><p><strong>colMean</strong> – Matrix X</p></li>
<li><p><strong>k</strong> – a constant used to discern outliers k*IQR</p></li>
<li><p><strong>isIterative</strong> – iterative repair or single repair</p></li>
<li><p><strong>repairMethod</strong> – values: 0 = delete rows having outliers,
1 = replace outliers with zeros
2 = replace outliers as missing values</p></li>
<li><p><strong>max_iterations</strong> – values: 0 = arbitrary number of iteraition until all outliers are removed,
n = any constant defined by user</p></li>
<li><p><strong>verbose</strong> – flag specifying if logging information should be printed</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix X with no outliers</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.pca">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">pca</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.pca" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin defines PCA that is a technique typically used to
reduce the number of dimensions of a matrix.
This implementation is based on calculating eigenvectors on
the covariance matrix of the input.</p>
<p>An example of calling in DML:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>data = read($1)
[data_reduced, Components] = pca(data=data, K=4, onlyComponents=TRUE)
print(Components)
</pre></div>
</div>
<p>An example in a ML pipeline containing PCA:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>X = read($1)
[X_reduced, Components] = pca(data=X, K=4)
Y = read($2)
bias = l2svm(X=X, Y=Y)
X_test = read($3)
[y_predict_normal, Y_predict_rounded] = l2svmPredict(X=X_test, W=bias)
write($5, Y_predict_rounded)
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>K</strong> – Number of components returned</p></li>
<li><p><strong>center</strong> – Indicates whether or not to center the feature matrix</p></li>
<li><p><strong>scale</strong> – Indicates whether or not to scale the feature matrix</p></li>
<li><p><strong>onlyComponents</strong> – Indicate if only the components should be calculated and returned
not the application of the components on X</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output feature matrix with K columns</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Output dominant eigen vectors sorted by influence</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The column means of the input, subtracted to construct the PCA</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>The scaling of the values, to make each dimension same size.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.pcaInverse">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">pcaInverse</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Clusters</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Centering</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ScaleFactor</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.pcaInverse" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Principal Component Analysis (PCA) for reconstruction of approximation of the original data.
This methods allows to reconstruct an approximation of the original matrix, and is useful for
calculating how much information is lost in the PCA.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>Y</strong> – Input features that have PCA applied to them</p></li>
<li><p><strong>Clusters</strong> – The previous PCA components computed</p></li>
<li><p><strong>Centering</strong> – The column means of the PCA model, subtracted to construct the PCA</p></li>
<li><p><strong>ScaleFactor</strong> – The scaling of each dimension in the PCA model</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output feature matrix reconstructing and approximation of the original matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.pcaTransform">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">pcaTransform</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Clusters</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Centering</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ScaleFactor</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.pcaTransform" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Principal Component Analysis (PCA) for dimensionality reduction prediction
This method is used to transpose data, which the PCA model was not trained on. To validate how good
The PCA is, and to apply in production.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>Clusters</strong> – The previously computed principal components</p></li>
<li><p><strong>Centering</strong> – The column means of the PCA model, subtracted to construct the PCA</p></li>
<li><p><strong>ScaleFactor</strong> – The scaling of each dimension in the PCA model</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output feature matrix dimensionally reduced by PCA</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.pnmf">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">pnmf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">rnk</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.pnmf" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The pnmf-function implements Poisson Non-negative Matrix Factorization (PNMF). Matrix X is factorized into two
non-negative matrices, W and H based on Poisson probabilistic assumption. This non-negativity makes the resulting
matrices easier to inspect.</p>
<p>[Chao Liu, Hung-chih Yang, Jinliang Fan, Li-Wei He, Yi-Min Wang:
Distributed nonnegative matrix factorization for web-scale dyadic 
data analysis on mapreduce. WWW 2010: 681-690]</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors.</p></li>
<li><p><strong>rnk</strong> – Number of components into which matrix X is to be factored.</p></li>
<li><p><strong>eps</strong> – Tolerance</p></li>
<li><p><strong>maxi</strong> – Maximum number of conjugate gradient iterations.</p></li>
<li><p><strong>verbose</strong> – If TRUE, ‘iter’ and ‘obj’ are printed.</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>List of pattern matrices, one for each repetition.</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>List of amplitude matrices, one for each repetition.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.ppca">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">ppca</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.ppca" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script performs Probabilistic Principal Component Analysis (PCA) on the given input data.
It is based on paper: sPCA: Scalable Principal Component Analysis for Big Data on Distributed
Platforms. Tarek Elgamal et.al.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – n x m input feature matrix</p></li>
<li><p><strong>k</strong> – indicates dimension of the new vector space constructed from eigen vectors</p></li>
<li><p><strong>maxi</strong> – maximum number of iterations until convergence</p></li>
<li><p><strong>tolobj</strong> – objective function tolerance value to stop ppca algorithm</p></li>
<li><p><strong>tolrecerr</strong> – reconstruction error tolerance value to stop the algorithm</p></li>
<li><p><strong>verbose</strong> – verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output feature matrix with K columns</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Output dominant eigen vectors (can be used for projections)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.psnr">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">psnr</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.psnr" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the peak signal to noise ratio</p>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio">https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio</a></p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The peak signal to noise ratio</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.randomForest">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">randomForest</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ctypes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.randomForest" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script implements random forest for recoded and binned categorical and
numerical input features. In detail, we train multiple CART (classification
and regression trees) decision trees in parallel and use them as an ensemble.
classifier/regressor. Each tree is trained on a sample of observations (rows)
and optionally subset of features (columns). During tree construction, split
candidates are additionally chosen on a sample of remaining features.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">For</span> <span class="n">example</span><span class="p">,</span> <span class="n">given</span> <span class="n">a</span> <span class="n">feature</span> <span class="n">matrix</span> <span class="k">with</span> <span class="n">features</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">,</span><span class="n">c</span><span class="p">,</span><span class="n">d</span><span class="p">]</span>
<span class="ow">and</span> <span class="n">the</span> <span class="n">following</span> <span class="n">two</span> <span class="n">trees</span><span class="p">,</span> <span class="n">M</span> <span class="p">(</span><span class="n">the</span> <span class="n">output</span><span class="p">)</span> <span class="n">would</span> <span class="n">look</span> <span class="k">as</span> <span class="n">follows</span><span class="p">:</span>

<span class="p">(</span><span class="n">L1</span><span class="p">)</span>          <span class="o">|</span><span class="n">a</span><span class="o">&lt;</span><span class="mi">7</span><span class="o">|</span>                   <span class="o">|</span><span class="n">d</span><span class="o">&lt;</span><span class="mi">5</span><span class="o">|</span>
             <span class="o">/</span>     \                 <span class="o">/</span>            <span class="p">(</span><span class="n">L2</span><span class="p">)</span>     <span class="o">|</span><span class="n">c</span><span class="o">&lt;</span><span class="mi">3</span><span class="o">|</span>     <span class="o">|</span><span class="n">b</span><span class="o">&lt;</span><span class="mi">4</span><span class="o">|</span>         <span class="o">|</span><span class="n">a</span><span class="o">&lt;</span><span class="mi">7</span><span class="o">|</span>     <span class="n">P3</span><span class="p">:</span><span class="mi">2</span>
         <span class="o">/</span>   \     <span class="o">/</span>   \         <span class="o">/</span>          <span class="p">(</span><span class="n">L3</span><span class="p">)</span>   <span class="n">P1</span><span class="p">:</span><span class="mi">2</span> <span class="n">P2</span><span class="p">:</span><span class="mi">1</span> <span class="n">P3</span><span class="p">:</span><span class="mi">1</span> <span class="n">P4</span><span class="p">:</span><span class="mi">2</span>     <span class="n">P1</span><span class="p">:</span><span class="mi">2</span> <span class="n">P2</span><span class="p">:</span><span class="mi">1</span>
<span class="o">--&gt;</span> <span class="n">M</span> <span class="o">:=</span>
<span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>  <span class="p">(</span><span class="mi">1</span><span class="n">st</span> <span class="n">tree</span><span class="p">)</span>
 <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">]]</span>  <span class="p">(</span><span class="mi">2</span><span class="n">nd</span> <span class="n">tree</span><span class="p">)</span>
 <span class="o">|</span><span class="p">(</span><span class="n">L1</span><span class="p">)</span><span class="o">|</span> <span class="o">|</span>  <span class="p">(</span><span class="n">L2</span><span class="p">)</span>   <span class="o">|</span> <span class="o">|</span>        <span class="p">(</span><span class="n">L3</span><span class="p">)</span>         <span class="o">|</span>

<span class="n">With</span> <span class="n">feature</span> <span class="n">sampling</span> <span class="p">(</span><span class="n">feature_frac</span> <span class="o">&lt;</span> <span class="mi">1</span><span class="p">),</span> <span class="n">each</span> <span class="n">tree</span> <span class="ow">is</span>
<span class="n">prefixed</span> <span class="n">by</span> <span class="n">a</span> <span class="n">one</span><span class="o">-</span><span class="n">hot</span> <span class="n">vector</span> <span class="n">of</span> <span class="n">sampled</span> <span class="n">features</span>
<span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">g</span><span class="o">.</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">0</span><span class="p">]</span> <span class="k">if</span> <span class="n">we</span> <span class="n">sampled</span> <span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">,</span><span class="n">c</span> <span class="n">of</span> <span class="n">the</span> <span class="n">four</span> <span class="n">features</span><span class="p">)</span>
</pre></div>
</div>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix in recoded/binned representation</p></li>
<li><p><strong>y</strong> – Label matrix in recoded/binned representation</p></li>
<li><p><strong>ctypes</strong> – Row-Vector of column types [1 scale/ordinal, 2 categorical]
of shape 1-by-(ncol(X)+1), where the last entry is the y type</p></li>
<li><p><strong>num_trees</strong> – Number of trees to be learned in the random forest model</p></li>
<li><p><strong>sample_frac</strong> – Sample fraction of examples for each tree in the forest</p></li>
<li><p><strong>feature_frac</strong> – Sample fraction of features for each tree in the forest</p></li>
<li><p><strong>max_depth</strong> – Maximum depth of the learned tree (stopping criterion)</p></li>
<li><p><strong>min_leaf</strong> – Minimum number of samples in leaf nodes (stopping criterion)</p></li>
<li><p><strong>min_split</strong> – Minimum number of samples in leaf for attempting a split</p></li>
<li><p><strong>max_features</strong> – Parameter controlling the number of features used as split
candidates at tree nodes: m = ceil(num_features^max_features)</p></li>
<li><p><strong>max_values</strong> – Parameter controlling the number of values per feature used
as split candidates: nb = ceil(num_values^max_values)</p></li>
<li><p><strong>impurity</strong> – Impurity measure: entropy, gini (default), rss (regression)</p></li>
<li><p><strong>seed</strong> – Fixed seed for randomization of samples and split candidates</p></li>
<li><p><strong>verbose</strong> – Flag indicating verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix M containing the learned trees, in linearized form.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.randomForestPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">randomForestPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ctypes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">M</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.randomForestPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script implements random forest prediction for recoded and binned
categorical and numerical input features.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix in recoded/binned representation</p></li>
<li><p><strong>y</strong> – Label matrix in recoded/binned representation,
optional for accuracy evaluation</p></li>
<li><p><strong>ctypes</strong> – Row-Vector of column types [1 scale/ordinal, 2 categorical]</p></li>
<li><p><strong>M</strong> – Matrix M holding the learned trees (one tree per row),
see randomForest() for the detailed tree representation.</p></li>
<li><p><strong>verbose</strong> – Flag indicating verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Label vector of predictions</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.rmse">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">rmse</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.rmse" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the root means square error between the two inputs</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The root means square error</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.scale">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">scale</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.scale" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function scales and center individual features in the input
matrix (column wise.) using z-score to scale the values.
The transformation is sometimes also called scale and shift,
but it is shifted first and then subsequently scaled.</p>
<p>The method is not resistant to inputs containing NaN nor overflows
of doubles, but handle it by guaranteeing that no extra NaN values
are introduced and columns that contain NaN will not be scaled or shifted.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>center</strong> – Indicates to center the feature matrix</p></li>
<li><p><strong>scale</strong> – Indicates to scale the feature matrix according to z-score</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output feature matrix scaled and shifted</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>The column means of the input, subtracted if Center was TRUE</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The scaling of the values, to make each dimension have similar value ranges</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.scaleApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">scaleApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Centering</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ScaleFactor</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.scaleApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function scales and center individual features in the input matrix (column wise.) using the input matrices.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>Centering</strong> – The column means to subtract from X (not done if empty)</p></li>
<li><p><strong>ScaleFactor</strong> – The column scaling to multiply with X (not done if empty)</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output feature matrix with K columns</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.scaleMinMax">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">scaleMinMax</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.scaleMinMax" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function performs min-max normalization (rescaling to [0,1]).</p>
<p>This function is deprecated, use normalize instead.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – Input feature matrix</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Scaled output matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.selectByVarThresh">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">selectByVarThresh</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.selectByVarThresh" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function drops feature with &lt;= thresh variance (by default drop constants).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors.</p></li>
<li><p><strong>thresh</strong> – The threshold for to drop</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix of feature vectors with &lt;= thresh variance.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.setdiff">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">setdiff</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.setdiff" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that implements difference operation on vectors</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – input vector</p></li>
<li><p><strong>Y</strong> – input vector</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>vector with all elements that are present in X but not in Y</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.sherlock">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">sherlock</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X_train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y_train</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.sherlock" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function implements training phase of Sherlock: A Deep Learning Approach to Semantic Data Type Detection</p>
<p>[Hulsebos, Madelon, et al. “Sherlock: A deep learning approach to semantic data type detection.”
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &amp; Data Mining.
2019.]</p>
<p>Split feature matrix into four different feature categories and train neural networks on the
respective single features. Then combine all trained features to train final neural network.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X_train</strong> – matrix of feature vectors</p></li>
<li><p><strong>y_train</strong> – matrix Y of class labels of semantic data type</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>weights (parameters) matrices for character distributions</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>biases vectors for character distributions</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>weights (parameters) matrices for word embeddings</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>biases vectors for word embeddings</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>weights (parameters) matrices for paragraph vectors</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>biases vectors for paragraph vectors</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>weights (parameters) matrices for global statistics</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>biases vectors for global statistics</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>weights (parameters) matrices for  combining all trained features (final)</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>biases vectors for combining all trained features (final)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.sherlockPredict">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">sherlockPredict</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cW1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cb1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cW2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cb2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cW3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">cb3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">wW1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">wb1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">wW2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">wb2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">wW3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">wb3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pW1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pb1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pW2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pb2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pW3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">pb3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sW1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sb1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sW2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sb2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sW3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sb3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">fW1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">fb1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">fW2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">fb2</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">fW3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">fb3</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.sherlockPredict" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function implements prediction and evaluation phase of Sherlock:
Split feature matrix into four different feature categories and predicting the class probability
on the respective features. Then combine all predictions for final predicted probabilities.
A Deep Learning Approach to Semantic Data Type Detection.
[Hulsebos, Madelon, et al. “Sherlock: A deep learning approach to semantic data type detection.”
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &amp; Data Mining.
2019.]</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – matrix of values which are to be classified</p></li>
<li><p><strong>cW</strong> – weights (parameters) matrices for character distribtions</p></li>
<li><p><strong>cb</strong> – biases vectors for character distribtions</p></li>
<li><p><strong>wW</strong> – weights (parameters) matrices for word embeddings</p></li>
<li><p><strong>wb</strong> – biases vectors for word embeddings</p></li>
<li><p><strong>pW</strong> – weights (parameters) matrices for paragraph vectors</p></li>
<li><p><strong>pb</strong> – biases vectors for paragraph vectors</p></li>
<li><p><strong>sW</strong> – weights (parameters) matrices for global statistics</p></li>
<li><p><strong>sb</strong> – biases vectors for global statistics</p></li>
<li><p><strong>fW</strong> – weights (parameters) matrices for  combining all trained features (final)</p></li>
<li><p><strong>fb</strong> – biases vectors for combining all trained features (final)</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>class probabilities of shape (N, K)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.shortestPath">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">shortestPath</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">G</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">sourceNode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.shortestPath" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Computes the minimum distances (shortest-path) between a single source vertex and every other vertex in the graph.</p>
<p>Grzegorz Malewicz, Matthew H. Austern, Aart J. C. Bilk, 
James C. Dehnert, Ikkan Horn, Naty Leiser and Grzegorz Czajkowski:
Pregel: A System for Large-Scale Graph Processing, SIGMOD 2010</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>G</strong> – adjacency matrix of the labeled graph: Such graph can be directed
(G is symmetric) or undirected (G is not symmetric).
The values of G can be 0/1 (just specifying whether the nodes
are connected or not) or integer values (representing the weight
of the edges or the distances between nodes, 0 if not connected).</p></li>
<li><p><strong>maxi</strong> – Integer max number of iterations accepted (0 for FALSE, i.e.
max number of iterations not defined)</p></li>
<li><p><strong>sourceNode</strong> – node index to calculate the shortest paths to all other nodes.</p></li>
<li><p><strong>verbose</strong> – flag for verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output matrix (double) of minimum distances (shortest-path) between
vertices: The value of the ith row and the jth column of the output
matrix is the minimum distance shortest-path from vertex i to vertex j.
When the value of the minimum distance is infinity, the two nodes are
not connected.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.sigmoid">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">sigmoid</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.sigmoid" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The Sigmoid function is a type of activation function, and also defined as a squashing function which limit the
output to a range between 0 and 1, which will make these functions useful in the prediction of probabilities.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – Matrix of feature vectors.</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>1-column matrix of weights.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.skewness">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">skewness</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.skewness" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the skewness of the matrix input</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – The matrix input</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The skewness of the input matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.slicefinder">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">slicefinder</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">e</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.slicefinder" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function implements SliceLine, a linear-algebra-based
ML model debugging technique for finding the top-k data slices where
a trained models performs significantly worse than on the overall 
dataset. For a detailed description and experimental results, see:
Svetlana Sagadeeva, Matthias Boehm: SliceLine: Fast, Linear-Algebra-based Slice Finding for ML Model Debugging.(SIGMOD 2021)</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix in recoded/binned representation</p></li>
<li><p><strong>e</strong> – Error vector of trained model</p></li>
<li><p><strong>k</strong> – Number of subsets required</p></li>
<li><p><strong>maxL</strong> – maximum level L (conjunctions of L predicates), 0 unlimited</p></li>
<li><p><strong>minSup</strong> – minimum support (min number of rows per slice)</p></li>
<li><p><strong>alpha</strong> – weight [0,1]: 0 only size, 1 only error</p></li>
<li><p><strong>tpEval</strong> – flag for task-parallel slice evaluation,
otherwise data-parallel</p></li>
<li><p><strong>tpBlksz</strong> – block size for task-parallel execution (num slices)</p></li>
<li><p><strong>selFeat</strong> – flag for removing one-hot-encoded features that don’t satisfy
the initial minimum-support constraint and/or have zero error</p></li>
<li><p><strong>verbose</strong> – flag for verbose debug output</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>top-k slices (k x ncol(X) if successful)</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>score, size, error of slices (k x 3)</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>debug matrix, populated with enumeration stats if verbose</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.smape">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">smape</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.smape" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Returns the symmetric means absolute percentage error between the two inputs</p>
<p>Monash Time Series Forecasting Archive
Rakshitha Godahewaa, Christoph Bergmeira, Geoffrey I. Webba, Rob J. Hyndmanb,
Pablo Montero-Mansoc</p>
<p>Another Look at Measures of Forecast Accuracy, R. J. Hyndman and A. B. Koehler, 2006.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – First Matrix to compare</p></li>
<li><p><strong>Y</strong> – Second Matrix to compare</p></li>
<li><p><strong>P</strong> – Quantiles to extract as well if empty matrix not calculated</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The symmetric mean absolute percentage error</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Quantiles calculated</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.smote">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">smote</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.smote" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function for handing class imbalance using Synthetic Minority Over-sampling Technique (SMOTE)
by Nitesh V. Chawla et. al. In Journal of Artificial Intelligence Research 16 (2002). 321–357</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of minority class samples</p></li>
<li><p><strong>mask</strong> – 0/1 mask vector where 0 represent numeric value and 1 represent categorical value</p></li>
<li><p><strong>s</strong> – Amount of SMOTE (percentage of oversampling), integral multiple of 100</p></li>
<li><p><strong>k</strong> – Number of nearest neighbor</p></li>
<li><p><strong>verbose</strong> – if the algorithm should be verbose</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix of (N/100)-1 * nrow(X) synthetic minority class samples</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.softmax">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">softmax</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">S</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.softmax" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Performs softmax on the given input matrix.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>S</strong> – Inputs of shape (N, D).</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Outputs of shape (N, D).</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.split">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">split</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.split" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function split input data X and Y into contiguous or samples train/test sets</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>Y</strong> – Input Labels</p></li>
<li><p><strong>f</strong> – Train set fraction [0,1]</p></li>
<li><p><strong>cont</strong> – contiguous splits, otherwise sampled</p></li>
<li><p><strong>seed</strong> – The seed to randomly select rows in sampled mode</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Train split of feature matrix</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Test split of feature matrix</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Train split of label matrix</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Test split of label matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.splitBalanced">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">splitBalanced</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.splitBalanced" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This functions split input data X and Y into contiguous balanced ratio
Related to [SYSTEMDS-2902] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>Y</strong> – Input Labels</p></li>
<li><p><strong>f</strong> – Train set fraction [0,1]</p></li>
<li><p><strong>verbose</strong> – print available</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Train split of feature matrix</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Test split of feature matrix</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Train split of label matrix</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Test split of label matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.stableMarriage">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">stableMarriage</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">P</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">A</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.stableMarriage" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This script computes a solution for the stable marriage problem.</p>
<p>result description:</p>
<p>If cell [i,j] is non-zero, it means that acceptor i has matched with
proposer j. Further, if cell [i,j] is non-zero, it holds the preference
value that led to the match.
Proposers.mtx:
2.0,1.0,3.0
1.0,2.0,3.0
1.0,3.0,2.0</p>
<p>Since ordered=TRUE, this means that proposer 1 (row 1) likes acceptor 2
the most, followed by acceptor 1 and acceptor 3.
If ordered=FALSE, this would mean that proposer 1 (row 1) likes acceptor 3
the most (since the value at [1,3] is the row max),
followed by acceptor 1 (2.0 preference value) and acceptor 2 (1.0 preference value).</p>
<p>Acceptors.mtx:
3.0,1.0,2.0
2.0,1.0,3.0
3.0,2.0,1.0</p>
<p>Since ordered=TRUE, this means that acceptor 1 (row 1) likes proposer 3
the most, followed by proposer 1 and proposer 2.
If ordered=FALSE, this would mean that acceptor 1 (row 1) likes proposer 1
the most (since the value at [1,1] is the row max),
followed by proposer 3 (2.0 preference value) and proposer 2
(1.0 preference value).</p>
<p>Output.mtx (assuming ordered=TRUE):
0.0,0.0,3.0
0.0,3.0,0.0
1.0,0.0,0.0</p>
<p>Acceptor 1 has matched with proposer 3 (since [1,3] is non-zero) at a
preference level of 3.0.
Acceptor 2 has matched with proposer 2 (since [2,2] is non-zero) at a
preference level of 3.0.
Acceptor 3 has matched with proposer 1 (since [3,1] is non-zero) at a
preference level of 1.0.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>P</strong> – proposer matrix P.
It must be a square matrix with no zeros.</p></li>
<li><p><strong>A</strong> – acceptor matrix A.
It must be a square matrix with no zeros.</p></li>
<li><p><strong>ordered</strong> – If true, P and A are assumed to be ordered,
i.e. the leftmost value in a row is the most preferred partner’s index.
i.e. the leftmost value in a row in P is the preference value for the acceptor with
index 1 and vice-versa (higher is better).</p></li>
<li><p><strong>verbose</strong> – if the algorithm should print verbosely</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Result Matrix</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.statsNA">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">statsNA</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.statsNA" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The statsNA-function Print summary stats about the distribution of missing values in a univariate time series.</p>
<dl class="simple">
<dt>result matrix contains the following:</dt><dd><ol class="arabic simple">
<li><p>Length of time series (including NAs)</p></li>
<li><p>Number of Missing Values (NAs)</p></li>
<li><p>Percentage of Missing Values (#2/#1)</p></li>
<li><p>Number of Gaps (consisting of one or more consecutive NAs)</p></li>
<li><p>Average Gap Size - Average size of consecutive NAs for the NA gaps</p></li>
<li><p>Longest NA gap - Longest series of consecutive missing values</p></li>
<li><p>Most frequent gap size - Most frequently occurring gap size</p></li>
<li><p>Gap size accounting for most NAs</p></li>
</ol>
</dd>
</dl>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Numeric Vector (‘vector’) object containing NAs</p></li>
<li><p><strong>bins</strong> – Split number for bin stats. Number of bins the time series gets
divided into. For each bin information about amount/percentage of
missing values is printed.</p></li>
<li><p><strong>verbose</strong> – Print detailed information.
For print_only = TRUE, the missing value stats are printed with
more information (“Stats for Bins” and “overview NA series”).</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Column vector where each row correspond to described values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.steplm">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">steplm</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.steplm" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The steplm-function (stepwise linear regression) implements a classical forward feature selection method.
This method iteratively runs what-if scenarios and greedily selects the next best feature
until the Akaike information criterion (AIC) does not improve anymore. Each configuration trains a regression model
via lm, which in turn calls either the closed form lmDS or iterative lmGC.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">return</span><span class="p">:</span> <span class="n">Matrix</span> <span class="n">of</span> <span class="n">regression</span> <span class="n">parameters</span> <span class="p">(</span><span class="n">the</span> <span class="n">betas</span><span class="p">)</span> <span class="ow">and</span> <span class="n">its</span> <span class="n">size</span> <span class="n">depend</span> <span class="n">on</span> <span class="n">icpt</span> <span class="nb">input</span> <span class="n">value</span><span class="p">:</span>
        <span class="n">OUTPUT</span> <span class="n">SIZE</span><span class="p">:</span>   <span class="n">OUTPUT</span> <span class="n">CONTENTS</span><span class="p">:</span>                <span class="n">HOW</span> <span class="n">TO</span> <span class="n">PREDICT</span> <span class="n">Y</span> <span class="n">FROM</span> <span class="n">X</span> <span class="n">AND</span> <span class="n">B</span><span class="p">:</span>
<span class="n">icpt</span><span class="o">=</span><span class="mi">0</span><span class="p">:</span> <span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">)</span>   <span class="n">x</span> <span class="mi">1</span>  <span class="n">Betas</span> <span class="k">for</span> <span class="n">X</span> <span class="n">only</span>                <span class="n">Y</span> <span class="o">~</span> <span class="n">X</span> <span class="o">%*%</span> <span class="n">B</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">),</span> <span class="mi">1</span><span class="p">],</span> <span class="ow">or</span> <span class="n">just</span> <span class="n">X</span> <span class="o">%*%</span> <span class="n">B</span>
<span class="n">icpt</span><span class="o">=</span><span class="mi">1</span><span class="p">:</span> <span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span> <span class="n">x</span> <span class="mi">1</span>  <span class="n">Betas</span> <span class="k">for</span> <span class="n">X</span> <span class="ow">and</span> <span class="n">intercept</span>       <span class="n">Y</span> <span class="o">~</span> <span class="n">X</span> <span class="o">%*%</span> <span class="n">B</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">),</span> <span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="n">B</span><span class="p">[</span><span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span>
<span class="n">icpt</span><span class="o">=</span><span class="mi">2</span><span class="p">:</span> <span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span> <span class="n">x</span> <span class="mi">2</span>  <span class="n">Col</span><span class="mf">.1</span><span class="p">:</span> <span class="n">betas</span> <span class="k">for</span> <span class="n">X</span> <span class="o">&amp;</span> <span class="n">intercept</span>  <span class="n">Y</span> <span class="o">~</span> <span class="n">X</span> <span class="o">%*%</span> <span class="n">B</span><span class="p">[</span><span class="mi">1</span><span class="p">:</span><span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">),</span> <span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="n">B</span><span class="p">[</span><span class="n">ncol</span><span class="p">(</span><span class="n">X</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span>
                       <span class="n">Col</span><span class="mf">.2</span><span class="p">:</span> <span class="n">betas</span> <span class="k">for</span> <span class="n">shifted</span><span class="o">/</span><span class="n">rescaled</span> <span class="n">X</span> <span class="ow">and</span> <span class="n">intercept</span>
</pre></div>
</div>
<p>In addition, in the last run of linear regression some statistics are provided in CSV format, one comma-separated
name-value pair per each line, as follows:</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Location (on HDFS) to read the matrix X of feature vectors</p></li>
<li><p><strong>Y</strong> – Location (on HDFS) to read the 1-column matrix Y of response values</p></li>
<li><p><strong>icpt</strong> – Intercept presence, shifting and rescaling the columns of X:
0 = no intercept, no shifting, no rescaling;
1 = add intercept, but neither shift nor rescale X;
2 = add intercept, shift &amp; rescale X columns to mean = 0, variance = 1</p></li>
<li><p><strong>reg</strong> – learning rate</p></li>
<li><p><strong>tol</strong> – Tolerance threshold to train until achieved</p></li>
<li><p><strong>maxi</strong> – maximum iterations 0 means until tolerance is reached</p></li>
<li><p><strong>verbose</strong> – If the algorithm should be verbose</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix of regression parameters (the betas) and its size depend on icpt input value.</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Matrix of selected features ordered as computed by the algorithm.</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.stratstats">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">stratstats</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.stratstats" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The stratstats.dml script computes common bivariate statistics, such as correlation, slope, and their p-value,
in parallel for many pairs of input variables in the presence of a confounding categorical variable.</p>
<p>Output contains:
(1st covariante, 2nd covariante)
40 columns containing the following information:
Col 01: 1st covariate X-column number
Col 02: 1st covariate global presence count
Col 03: 1st covariate global mean
Col 04: 1st covariate global standard deviation
Col 05: 1st covariate stratified standard deviation
Col 06: R-squared, 1st covariate vs. strata
Col 07: adjusted R-squared, 1st covariate vs. strata
Col 08: P-value, 1st covariate vs. strata
Col 09-10: Reserved
Col 11: 2nd covariate Y-column number
Col 12: 2nd covariate global presence count
Col 13: 2nd covariate global mean
Col 14: 2nd covariate global standard deviation
Col 15: 2nd covariate stratified standard deviation
Col 16: R-squared, 2nd covariate vs. strata
Col 17: adjusted R-squared, 2nd covariate vs. strata
Col 18: P-value, 2nd covariate vs. strata
Col 19-20: Reserved
Col 21: Global 1st &amp; 2nd covariate presence count
Col 22: Global regression slope (2nd vs. 1st covariate)
Col 23: Global regression slope standard deviation
Col 24: Global correlation = +/- sqrt(R-squared)
Col 25: Global residual standard deviation
Col 26: Global R-squared
Col 27: Global adjusted R-squared
Col 28: Global P-value for hypothesis “slope = 0”
Col 29-30: Reserved
Col 31: Stratified 1st &amp; 2nd covariate presence count
Col 32: Stratified regression slope (2nd vs. 1st covariate)
Col 33: Stratified regression slope standard deviation
Col 34: Stratified correlation = +/- sqrt(R-squared)
Col 35: Stratified residual standard deviation
Col 36: Stratified R-squared
Col 37: Stratified adjusted R-squared
Col 38: Stratified P-value for hypothesis “slope = 0”
Col 39: Number of strata with at least two counted points
Col 40: Reserved</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix X that has all 1-st covariates</p></li>
<li><p><strong>Y</strong> – Matrix Y that has all 2-nd covariates
the default value empty means “use X in place of Y”</p></li>
<li><p><strong>S</strong> – Matrix S that has the stratum column
the default value empty means “use X in place of S”</p></li>
<li><p><strong>Xcid</strong> – 1-st covariate X-column indices
the default value empty means “use columns 1 : ncol(X)”</p></li>
<li><p><strong>Ycid</strong> – 2-nd covariate Y-column indices
the default value empty means “use columns 1 : ncol(Y)”</p></li>
<li><p><strong>Scid</strong> – Column index of the stratum column in S</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Output matrix, one row per each distinct pair</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.symmetricDifference">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">symmetricDifference</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.symmetricDifference" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that implements symmetric difference set-operation on vectors</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – input vector</p></li>
<li><p><strong>Y</strong> – input vector</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>vector with all elements in X and Y but not in both</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.tSNE">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">tSNE</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.tSNE" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This function performs dimensionality reduction using tSNE algorithm based on
the paper: Visualizing Data using t-SNE, Maaten et. al.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix of shape
(number of data points, input dimensionality)</p></li>
<li><p><strong>reduced_dims</strong> – Output dimensionality</p></li>
<li><p><strong>perplexity</strong> – Perplexity Parameter</p></li>
<li><p><strong>lr</strong> – Learning rate</p></li>
<li><p><strong>momentum</strong> – Momentum Parameter</p></li>
<li><p><strong>max_iter</strong> – Number of iterations</p></li>
<li><p><strong>seed</strong> – The seed used for initial values.
If set to -1 random seeds are selected.</p></li>
<li><p><strong>is_verbose</strong> – Print debug information</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Data Matrix of shape (number of data points, reduced_dims)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.toOneHot">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">toOneHot</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">numClasses</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.toOneHot" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The toOneHot-function encodes unordered categorical vector to multiple binary vectors.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Vector with N integer entries between 1 and numClasses</p></li>
<li><p><strong>numclasses</strong> – Number of columns, must be be greater than or equal to largest value in X</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>One-hot-encoded matrix with shape (N, numClasses)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.tomeklink">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">tomeklink</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.tomeklink" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The tomekLink-function performs under sampling by removing Tomek’s links for imbalanced multi-class problems
Computes TOMEK links and drops them from data matrix and label vector.
Drops only the majority label and corresponding point of TOMEK links.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Data Matrix (nxm)</p></li>
<li><p><strong>y</strong> – Label Matrix (nx1), greater than zero</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Data Matrix without Tomek links</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>Labels corresponding to under sampled data</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Indices of dropped rows/labels wrt input</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.topk_cleaning">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">topk_cleaning</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">dataTrain</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">primitives</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">parameters</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/frame.html#systemds.operator.Frame" title="systemds.operator.nodes.frame.Frame"><span class="pre">Frame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">evaluationFunc</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">evalFunHp</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.topk_cleaning" title="Link to this definition"></a></dt>
<dd><p>This function cleans top-K item (where K is given as input)for a given list of users.
metaData[3, ncol(X)] : metaData[1] stores mask, metaData[2] stores schema, metaData[3] stores FD mask</p>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.underSampling">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">underSampling</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">ratio</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.underSampling" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin to perform random under sampling on data.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – X data to sample from</p></li>
<li><p><strong>Y</strong> – Y data to sample from it will sample the same rows from x.</p></li>
<li><p><strong>ratio</strong> – The ratio to sample</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The under sample data X</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p>The under sample data Y</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.union">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">union</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">Y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.union" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Builtin function that implements union operation on vectors</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – input vector</p></li>
<li><p><strong>Y</strong> – input vector</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>matrix with all unique rows existing in X and Y</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.univar">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">univar</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">types</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.univar" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>Computes univariate statistics for all attributes in a given data set</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input matrix of the shape (N, D)</p></li>
<li><p><strong>TYPES</strong> – Matrix of the shape (1, D) with features types:
1 for scale, 2 for nominal, 3 for ordinal</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>univariate statistics for all attributes</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.vectorToCsv">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">vectorToCsv</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">mask</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.vectorToCsv" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function  convert vector into csv string such as [1 0 0 1 1 0 1] = “1,4,5,7”
Related to [SYSTEMDS-2662] dependency function for cleaning pipelines</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>mask</strong> – Data vector (having 0 for excluded indexes)</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>indexes</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.winsorize">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">winsorize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">verbose</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.winsorize" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>The winsorize-function removes outliers from the data. It does so by computing upper and
lower quartile range of the given data then it replaces any value that falls outside this range
(less than lower quartile range or more than upper quartile range).</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>verbose</strong> – To print output on screen</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix without outlier values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.winsorizeApply">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">winsorizeApply</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">qLower</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">qUpper</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.winsorizeApply" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>winsorizeApply takes the upper and lower quantile values per column, and
remove outliers by replacing them with these upper and lower bound values.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Input feature matrix</p></li>
<li><p><strong>qLower</strong> – row vector of upper bounds per column</p></li>
<li><p><strong>qUpper</strong> – row vector of lower bounds per column</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix without outlier values</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.xdummy1">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">xdummy1</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.xdummy1" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function is here for debugging purposes</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – test input</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>test result</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.xdummy2">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">xdummy2</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.xdummy2" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>This builtin function is here for debugging purposes</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><p><strong>X</strong> – Debug input</p>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p><p>—</p>
</p>
</dd>
<dt class="field-odd">Returns<span class="colon">:</span></dt>
<dd class="field-odd"><p><p>—</p>
</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.xgboost">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">xgboost</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">y</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.xgboost" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting. This xgboost
implementation supports classification and regression and is capable of working with categorical and scalar features.</p>
<p>Output explained:
(the first node is the init prediction) and each row contains
the following information:
M[1,j]: id of node j (in a complete binary tree)
M[2,j]: tree id to which node j belongs
M[3,j]: Offset (no. of columns) to left child of j if j is an internal node, otherwise 0
M[4,j]: Feature index of the feature (scale feature id if the feature is
scale or categorical feature id if the feature is categorical)
that node j looks at if j is an internal node, otherwise 0
M[5,j]: Type of the feature that node j looks at if j is an internal node.
if leaf = 0, if scalar = 1, if categorical = 2
M[6:,j]: If j is an internal node: Threshold the example’s feature value is
compared to is stored at M[6,j] if the feature chosen for j is scale,
otherwise if the feature chosen for j is categorical rows 6,7,… depict
the value subset chosen for j
If j is a leaf node 1 if j is impure and the number of samples at j &gt; threshold, otherwise 0</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Feature matrix X; note that X needs to be both recoded and dummy coded</p></li>
<li><p><strong>y</strong> – Label matrix y; note that y needs to be both recoded and dummy coded</p></li>
<li><p><strong>R</strong> – Matrix R; 1xn vector which for each feature in X contains the following information
- R[,1]: 1 (scalar feature)
- R[,2]: 2 (categorical feature)
Feature 1 is a scalar feature and features 2 is a categorical feature
If R is not provided by default all variables are assumed to be scale (1)</p></li>
<li><p><strong>sml_type</strong> – Supervised machine learning type: 1 = Regression(default), 2 = Classification</p></li>
<li><p><strong>num_trees</strong> – Number of trees to be created in the xgboost model</p></li>
<li><p><strong>learning_rate</strong> – Alias: eta. After each boosting step the learning rate controls the weights of the new predictions</p></li>
<li><p><strong>max_depth</strong> – Maximum depth of a tree. Increasing this value will make the model more complex and more likely to overfit</p></li>
<li><p><strong>lambda</strong> – L2 regularization term on weights. Increasing this value will make model more conservative and reduce amount of leaves of a tree</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>Matrix M where each column corresponds to a node in the learned tree</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.xgboostPredictClassification">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">xgboostPredictClassification</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">M</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.xgboostPredictClassification" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting. This xgboost
implementation supports classification  and is capable of working with categorical features.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors we want to predict (X_test)</p></li>
<li><p><strong>M</strong> – The model created at xgboost</p></li>
<li><p><strong>learning_rate</strong> – The learning rate used in the model</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The predictions of the samples using the given xgboost model. (y_prediction)</p>
</dd>
</dl>
</dd></dl>

<dl class="py function">
<dt class="sig sig-object py" id="systemds.operator.algorithm.xgboostPredictRegression">
<span class="sig-prename descclassname"><span class="pre">systemds.operator.algorithm.</span></span><span class="sig-name descname"><span class="pre">xgboostPredictRegression</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">X</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">M</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="node/matrix.html#systemds.operator.Matrix" title="systemds.operator.nodes.matrix.Matrix"><span class="pre">Matrix</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dict</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference internal" href="../script_building/dag.html#systemds.script_building.dag.DAGNode" title="systemds.script_building.dag.DAGNode"><span class="pre">DAGNode</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">int</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">float</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">bool</span><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#systemds.operator.algorithm.xgboostPredictRegression" title="Link to this definition"></a></dt>
<dd><blockquote>
<div><p>XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting. This xgboost
implementation supports regression.</p>
</div></blockquote>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><ul class="simple">
<li><p><strong>X</strong> – Matrix of feature vectors we want to predict (X_test)</p></li>
<li><p><strong>M</strong> – The model created at xgboost</p></li>
<li><p><strong>learning_rate</strong> – The learning rate used in the model</p></li>
</ul>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><p>The predictions of the samples using the given xgboost model. (y_prediction)</p>
</dd>
</dl>
</dd></dl>

</section>


           </div>
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../context/systemds_context.html" class="btn btn-neutral float-left" title="SystemDSContext" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="node/matrix.html" class="btn btn-neutral float-right" title="Matrix" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2023, Apache SystemDS.</p>
  </div>

  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
    provided by <a href="https://readthedocs.org">Read the Docs</a>.
   

</footer>
        </div>
      </div>
    </section>
  </div>
  <script>
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script> 

</body>
</html>